Camera corrector

ABSTRACT

Herein disclosed is a camera calibrating apparatus for calibrating an optical parameter of a camera mounted on an automotive vehicle, comprising: a first housing position information storing section ( 115 ) for storing first housing position information, a second housing position information storing section ( 116 ) for storing second housing position information, a first optical position information producing section ( 117 ) for producing first optical position information, a first optical position information storing section ( 118 ) for storing the first optical position information, a second optical position information producing section ( 120 ) for producing second optical position information, a second optical position information storing section ( 130 ) for storing the second optical position information, and a calibrating section ( 160 ) for calibrating the second optical position information stored in the second optical position information storing section ( 130 ) on the basis of image information obtained by a camera ( 110 ) in a second coordination system ( 102 ).

TECHNICAL FIELD OF THE INVENTION

The present invention relates to a camera calibrating apparatus, andmore particularly to a camera calibrating apparatus for calibrating acamera mounted on an automotive vehicle or the like.

DESCRIPTION OF THE RELATED ART

In recent years, there has been developed an imaging controllingapparatus, for example an electronic control unit (herein after simplyreferred to as “ECU”), which is used to be electrically connected to acamera mounted on an exterior surface of an automotive vehicle to detectthe position occupied by an object located on a road, in particular, inthe vicinity of the automotive vehicle on the basis of image informationobtained by the camera. When, in general, the camera is mounted on theautomotive vehicle in combination with the imaging controlling apparatusof this type, the calibration of the camera is performed by a cameracalibrating apparatus in order to specify optical parameters of thecamera.

Up until now, there have been proposed a wide variety of cameracalibrating apparatuses of this type, all of which are classified intotwo different types including a first type to calibrate the camerabefore that camera is mounted on the automotive vehicle, and a secondtype to calibrate the camera after that camera is mounted on theautomotive vehicle. For the purpose of facilitating an assembly work ofthe camera to the automotive vehicle, there is a growing demand for thecamera calibrating apparatus for calibrating the camera mounted on theautomotive vehicle in recent years.

As shown in FIGS. 39 to 41, the conventional camera calibratingapparatus 500 is electrically connected to the camera 510 as an imagingdevice. The camera 510 includes a housing unit 511 and an opticalsection 512 supported by the housing unit 511 to obtain imageinformation through the optical section 512.

The conventional camera calibrating apparatus 500 comprises a firsthousing position information storing section 515 for storing firsthousing position information indicative of the position occupied by thehousing unit 511 in the first coordination system 501 of a cameraproducing plant, and a second housing position information storingsection 516 for storing second housing position information indicativeof the position occupied by the housing unit 511 in the secondcoordination system 502 of an automotive vehicle producing plant.

The camera calibrating apparatus 500 is adapted to revise the camera 510in the camera producing plant by using a revising marker 505 located inthe first coordination system 501. Here, the term “the calibration ofthe camera 510” is intended to indicate the calculation of the accurateposition of the optical section 512 of the camera 510 mounted on theautomotive vehicle 508.

The conventional camera calibrating apparatus 500 further comprises afirst optical position information producing section 517 for producingfirst optical position information indicative of the position of theoptical section 512 in the first coordination system 501, and a firstoptical position information storing section 518 for storing the firstoptical position information produced by the first optical positioninformation producing section 517. The first optical positioninformation producing section 517 is adapted to produce optical positioninformation indicative of the position of the optical section 512 in thefirst coordination system 501 on the basis of the image information ofthe revising marker 505 obtained by the camera 510.

The conventional camera calibrating apparatus 500 further comprises asecond optical position information producing section 520 for producingsecond optical information indicative of the position of the opticalsection 512 in the second coordination system 502, and a second opticalposition information storing section 530 for storing the second opticalposition information produced by the second optical position informationproducing section 520.

The second optical position information producing section 520 is adaptedto produce optical position information indicative of the position ofthe optical section 512 in the second coordination system 502 from thesecond housing position information stored in the second housingposition information storing section 516 on the basis of the firsthousing position information stored in the first housing positioninformation storing section 515 and the first optical positioninformation stored in the first optical position information storingsection 518.

The conventional camera calibrating apparatus 500 thus constructed aspreviously mentioned is adapted to revise the camera in the cameraproducing plant by calculating the position of the optical section 512in the second coordination system 502, while the image controllingapparatus is adapted to detect the position occupied by an objectlocated on the road through the image information obtained by the camera510 mounted on the automotive vehicle 508.

The conventional camera calibrating apparatus as previously mentioned,however, encounters such a problem that the second optical positioninformation stored in the second optical position information storingsection is not calibrated. This leads to the fact that the position ofthe object located on the road cannot be accurately detected by theimage controlling apparatus if the camera is mounted on an inaccurateposition of the automotive vehicle.

It is, therefore, an object of the present invention to provide a cameracalibrating apparatus which can calibrate optical parameters of theoptical section of the camera mounted on the automotive vehicle or thelike.

DISCLOSURE OF THE INVENTION

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate position information of the opticalsection, comprising: first housing position information storing meansfor storing first housing position information indicative of theposition of the housing unit in the first coordination system; secondhousing position information storing means for storing second housingposition information indicative of the position of the housing unit inthe second coordination system; first optical position informationproducing means for producing first optical position informationindicative of the position of the optical section in the firstcoordination system on the basis of the image information obtained bythe imaging device in the first coordination system; first opticalposition information storing means for storing the first opticalposition information produced by the first optical position informationproducing means; second optical position information producing means forproducing second optical position information indicative of the positionof the optical section in the second coordination system from the secondhousing position information stored by the second housing positioninformation storing means on the basis of the first housing positioninformation stored by the first housing position information storingmeans and the first optical position information stored by the firstoptical position information storing means; second optical positioninformation storing means for storing the second optical positioninformation produced by the second optical position informationproducing means; and calibrating means for calibrating the secondoptical position information stored by the second optical positioninformation storing means on the basis of the image information obtainedby the imaging device in the second coordination system. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate parameters of theoptical section mounted on an automotive vehicle or the like.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate optical position information about theposition of the optical section, comprising: optical positioninformation storing means for storing optical position informationindicative of the position of the optical section to a predeterminedcoordinate system; and calibrating means for calibrating the opticalposition information stored by the optical position information storingmeans on the basis of the image information obtained by the imagingdevice on the predetermined coordinate system. The camera calibratingapparatus according to the present invention thus constructed aspreviously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate optical position information about theposition of the optical section, comprising: first housing positioninformation storing means for storing first housing position informationindicative of the position of the housing unit in the first coordinationsystem in which a revising marker is located; second housing positioninformation storing means for storing second housing positioninformation indicative of the position of the housing unit in the secondcoordination system in which a calibrating marker is located; firstoptical position information producing means for producing first opticalposition information indicative of the position of the optical sectionin the first coordination system on the basis of the image informationof the revising marker obtained by the imaging device; first opticalposition information storing means for storing the first opticalposition information produced by the first optical position informationproducing means; second optical position information producing means forproducing second optical position information indicative of the positionof the optical section in the second coordination system from the secondhousing position information stored by the second housing positioninformation storing means on the basis of the first housing positioninformation stored by the first housing position information storingmeans and the first optical position information stored by the firstoptical position information storing means; estimated locationinformation producing means for producing estimated location informationindicative of the position of the calibrating marker to an imagecoordination system of the imaging device on the basis of the secondoptical position information produced by the second optical positioninformation producing means; second optical position information storingmeans for storing the second optical position information produced bythe second optical position information producing means; estimatedlocation information storing means for storing the estimated locationinformation produced by the estimated location information estimatingmeans; and calibrating means for calibrating the second optical positioninformation stored by the second optical position information storingmeans on the basis of the image information of the calibrating markerobtained by the imaging device and the estimated location informationstored by the estimated location information storing means. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using a simple calibrating marker.

In the camera calibrating apparatus according to the present invention,the calibrating means includes: image location information extractingmeans for extracting an image location information indicative of animage location of the calibrating marker to the image coordinationsystem of the imaging device on the basis of the image information ofthe calibrating marker obtained by the imaging device; calibration valuecalculating means for calculating a calibration value of the secondoptical position information stored by the second optical positioninformation storing means on the basis of the image location informationextracted by the image location information extracting means and theestimated location information stored by the estimated locationinformation storing means; and optical position information calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis ofthe calibration value calculated by the calibration value calculatingmeans. The camera calibrating apparatus according to the presentinvention thus constructed as previously mentioned can calculatecalibration value of the second optical position information by using asimple calibrating marker.

In the camera calibrating apparatus according to the present invention,the calibrating means is adapted to calibrate a deviation of arotational component of the second optical position information. Thecamera calibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using a simple calibrating marker without takinginto account parallel shift component of the second optical positioninformation.

In the camera calibrating apparatus according to the present invention,the image location information extracting means includes: imagedisplaying means for displaying an image of the calibrating markerobtained by the imaging device, and image location specifying means forspecifying the image location of the calibrating marker on the image ofthe calibrating marker displayed by the image displaying means and toensure that the image location information is extracted. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can specify the image location ofthe calibrating marker to ensure that the image location information ofthe calibrating marker is extracted.

In the camera calibrating apparatus according to the present invention,the image location information extracting means includes: estimated areainformation storing means for storing estimated area informationindicative of the calibrating marker to the image coordination system ofthe imaging device; and image location searching means for searching theimage location of the calibrating marker from the image information ofthe calibrating marker obtained by the imaging device on the basis ofthe estimated area information stored by the estimated area informationstoring means and the estimated location information stored by theestimated location information storing means to ensure that the imagelocation information is extracted. The camera calibrating apparatusaccording to the present invention thus constructed as previouslymentioned can search the image location of the calibrating marker toensure that the image location information of the calibrating marker iseasily extracted.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate optical position information about theposition of the optical section, comprising: first housing positioninformation storing means for storing first housing position informationindicative of the position of the housing unit in the first coordinationsystem in which a revising marker is located; second housing positioninformation storing means for storing second housing positioninformation indicative of the position of the housing unit in the secondcoordination system; first optical position information producing meansfor producing first optical position information indicative of theposition of the optical section in the first coordination system on thebasis of the image information obtained by the imaging device andindicative of the revising marker; first optical position informationstoring means for storing the first optical position informationproduced by the first optical position information producing means;second optical position information producing means for producing secondoptical position information indicative of the position of the opticalsection in the second coordination system from the second housingposition information stored by the second housing position informationstoring means on the basis of the first housing position informationstored by the first housing position information storing means and thefirst optical position information stored by the first optical positioninformation storing means; second optical position information storingmeans for storing the second optical position information produced bythe second optical position information producing means; and calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis of amotion vector of the image information obtained by the imaging device inthe second coordination system. The camera calibrating apparatusaccording to the present invention thus constructed as previouslymentioned can calibrate the second optical position information by usingthe motion vector.

In the camera calibrating apparatus according to the present invention,the calibrating means includes: plane-projected image producing meansfor producing a plane-projected image from the image informationobtained by the imaging device in the second coordination system;plane-projected image dividing means for dividing the plane-projectedimage produced by the plane-projected image producing means into aplurality of image segments; motion vector extracting means forextracting the motion vector from the image segments divided by theplane-projected image dividing means; calibration value calculatingmeans for calculating a calibration value of the second optical positioninformation stored by the second optical position information storingmeans on the basis of the motion vector extracted by the motion vectorextracting means; and optical position information calibrating means forcalibrating the second optical position information stored by the secondoptical position information storing means on the basis of thecalibration value calculated by the calibration value calculating means.The camera calibrating apparatus according to the present invention thusconstructed as previously mentioned can easily extract the motion vectorfrom the image segments.

In the camera calibrating apparatus according to the present invention,a dividing marker located in the second coordination system in apredetermined relationship with the position of the housing unitrepresented by the second position information stored by the secondposition information storing means. The plane-projected image dividingmeans is adapted to divide the plane-projected image produced by theplane-projected image producing means into a plurality of image segmentson the basis of the image information of the dividing marker obtained bythe imaging device. The camera calibrating apparatus according to thepresent invention thus constructed as previously mentioned canaccurately divide the plane-projected image by using the dividingmarker.

In accordance with the present invention, there is provided a cameracalibrating apparatus to be operative in combination with an imagingdevice which includes a housing unit and an optical section supported bythe housing unit to obtain image information thorough the opticalsection, and adapted to calibrate position information of the opticalsection, comprising: first housing position information storing meansfor storing first housing position information indicative of theposition of the housing unit in the first coordination system in which arevising marker is located; second housing position information storingmeans for storing second housing position information indicative of theposition of the housing unit in the second coordination system in whichan automotive vehicle is located; first optical position informationproducing means for producing first optical position informationindicative of the position of the optical section in the firstcoordination system on the basis of the image information of therevising marker obtained by the imaging device; first optical positioninformation storing means for storing the first optical positioninformation produced by the first optical position information producingmeans; second optical position information producing means for producingsecond optical position information indicative of the position of theoptical section in the second coordination system from the secondhousing position information stored by the second housing positioninformation storing means on the basis of the first housing positioninformation stored by the first housing position information storingmeans and the first optical position information stored by the firstoptical position information storing means; estimated locationinformation producing means for producing estimated location informationindicative of the position of the automotive vehicle to an imagecoordination system of the imaging device on the basis of the secondoptical position information produced by the second optical positioninformation producing means; second optical position information storingmeans for storing the second optical position information produced bythe second optical position information producing means; estimatedlocation information storing means for storing the estimated locationinformation produced by the estimated location information estimatingmeans; and calibrating means for calibrating the second optical positioninformation stored by the second optical position information storingmeans on the basis of the image information of the automotive vehicleobtained by the imaging device and the estimated location informationstored by the estimated location information storing means. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using a portion of the automotive vehicle.

In the camera calibrating apparatus according to the present invention,the calibrating means includes: image location information extractingmeans for extracting an image location information indicative of animage location of the automotive vehicle in the image coordinationsystem of the imaging device on the basis of the image information ofthe automotive vehicle obtained by the imaging device; calibration valuecalculating means for calculating a calibration value of the secondoptical position information stored by the second optical positioninformation storing means on the basis of the image location informationextracted by the image location information extracting means and theestimated location information stored by the estimated locationinformation storing means; and optical position information calibratingmeans for calibrating the second optical position information stored bythe second optical position information storing means on the basis ofthe calibration value calculated by the calibration value calculatingmeans. The camera calibrating apparatus according to the presentinvention thus constructed as previously mentioned can calculate thecalibration value of the second optical position information by using aportion of the automotive vehicle.

In the camera calibrating apparatus according to the present invention,the calibration value calculating means includes: superimposing meansfor superimposing a profile line of the automotive vehicle representedby the image location information on a profile line of the automotivevehicle represented by the estimated location information; extractingmeans for extracting a plurality of points from the overlapped profilelines of the automotive vehicle superimposed by the superimposing means,and calculating means for calculating a calibration value of the secondoptical position information by comparing the points of the imagelocation information with the points of the estimated locationinformation. The camera calibrating apparatus according to the presentinvention thus constructed as previously mentioned can extract thepoints from the profile line of the automotive vehicle to ensure thatthe second optical position information is extracted.

In the camera calibrating apparatus according to the present invention,the imaging device is mounted on an automotive vehicle. The cameracalibrating apparatus according to the present invention thusconstructed as previously mentioned can calibrate parameters of theoptical section to accurately detect the position of an object locatedon the road even if the housing unit is mounted on the inaccurateposition of the automotive vehicle.

In accordance with the present invention, there is provided an imagingsystem comprises a camera calibrating apparatus. The imaging systemaccording to the present invention thus constructed as previouslymentioned can calibrate parameters of the optical section mounted on anautomotive vehicle or the like.

In accordance with the present invention, there is provided an imagingcontrol system comprises a camera calibrating apparatus. The imagingcontrol system according to the present invention thus constructed aspreviously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinationsystem; a second housing position information storing step of storingsecond housing position information indicative of the position of thehousing unit in the second coordination system; a first optical positioninformation producing step of producing first optical positioninformation indicative of the position of the optical section in thefirst coordination system on the basis of the image information obtainedby the imaging device in the first coordination system; a first opticalposition information storing step of storing the first optical positioninformation produced in the first optical position information producingstep; a second optical position information producing step of producingsecond optical position information indicative of the position of theoptical section in the second coordination system from the secondhousing position information stored in the second housing positioninformation storing step on the basis of the first housing positioninformation stored in the first housing position information storingstep and the first optical position information stored in the firstoptical position information storing step; a second optical positioninformation storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; and a calibrating step of calibrating the second opticalposition information stored in the second optical position informationstoring step on the basis of the image information obtained by theimaging device in the second coordination system. The camera calibratingmethod according to the present invention thus constructed as previouslymentioned can calibrate parameters of the optical section mounted on anautomotive vehicle or the like to accurately detect the position of anobject located on the road.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: an optical position informationstoring step of storing optical position information indicative of theposition of the optical section to a predetermined coordinate system;and a calibrating step of calibrating the optical position informationstored in the optical position information storing step on the basis ofthe image information obtained by the imaging device on thepredetermined coordinate system. The camera calibrating method accordingto the present invention thus constructed as previously mentioned cancalibrate parameters of the optical section mounted on an automotivevehicle or the like to accurately detect the position of an objectlocated on the road.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinationsystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of the position of the housing unit in the secondcoordination system in which a calibrating marker is located; a firstoptical position information producing step of producing first opticalposition information indicative of the position of the optical sectionin the first coordination system on the basis of the image informationof the revising marker obtained by the imaging device; a first opticalposition information storing step of storing the first optical positioninformation produced in the first optical position information producingstep; a second optical position information producing step of producingsecond optical position information indicative of the position of theoptical section in the second coordination system from the secondhousing position information stored in the second housing positioninformation storing step on the basis of the first housing positioninformation stored in the first housing position information storingstep and the first optical position information stored in the firstoptical position information storing step; an estimated locationinformation producing step of producing estimated location informationindicative of the position of the calibrating marker to an imagecoordination system of the imaging device on the basis of the secondoptical position information produced in the second optical positioninformation producing step; a second optical position informationstoring step of storing the second optical position information producedin the second optical position information producing step; an estimatedlocation information storing step of storing the estimated locationinformation produced in the estimated location information estimatingstep; and a calibrating step of calibrating the second optical positioninformation stored in the second optical position information storingstep on the basis of the image information of the calibrating markerobtained by the imaging device and the estimated location informationstored in the estimated location information storing step. The cameracalibrating method according to the present invention thus constructedas previously mentioned can calibrate the second optical positioninformation by using the simple calibrating marker.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinationsystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of the position of the housing unit in the secondcoordination system; a first optical position information producing stepof producing first optical position information indicative of theposition of the optical section in the first coordination system on thebasis of the image information obtained by the imaging device andindicative of the revising marker; a first optical position informationstoring step of storing the first optical position information producedin the first optical position information producing step; a secondoptical position information producing step of producing second opticalposition information indicative of the position of the optical sectionin the second coordination system from the second housing positioninformation stored in the second housing position information storingstep on the basis of the first housing position information stored inthe first housing position information storing step and the firstoptical position information stored in the first optical positioninformation storing step; a second optical position information storingstep of storing the second optical position information produced in thesecond optical position information producing step; and a calibratingstep of calibrating the second optical position information stored inthe second optical position information storing step on the basis of amotion vector of the image information obtained by the imaging device inthe second coordination system. The camera calibrating method accordingto the present invention thus constructed as previously mentioned cancalibrate the second optical position information by using the motionvector.

In accordance with the present invention, there is provided a cameracalibrating method of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of the position of the housing unit in the first coordinationsystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of the position of the housing unit in the secondcoordination system in which an automotive vehicle is located; a firstoptical position information producing step of producing first opticalposition information indicative of the position of the optical sectionin the first coordination system on the basis of the image informationof the revising marker obtained by the imaging device; a first opticalposition information storing step of storing the first optical positioninformation produced in the first optical position information producingstep; a second optical position information producing step of producingsecond optical position information indicative of the position of theoptical section in the second coordination system from the secondhousing position information stored in the second housing positioninformation storing step on the basis of the first housing positioninformation stored in the first housing position information storingstep and the first optical position information stored in the firstoptical position information storing step; an estimated locationinformation producing step of producing estimated location informationindicative of the position of the automotive vehicle to an imagecoordination system of the imaging device on the basis of the secondoptical position information produced in the second optical positioninformation producing step; a second optical position informationstoring step of storing the second optical position information producedin the second optical position information producing step; an estimatedlocation information storing step of storing the estimated locationinformation produced in the estimated location information estimatingstep; and a calibrating step of calibrating the second optical positioninformation stored in the second optical position information storingstep on the basis of the image information of the automotive vehicleobtained by the imaging device and the estimated location informationstored in the estimated location information storing step. The cameracalibrating method according to the present invention thus constructedas previously mentioned can calibrate the second optical positioninformation by using a portion of the automotive vehicle.

In the camera calibrating apparatus according to the present invention,the imaging device is mounted on an automotive vehicle. The cameracalibrating method according to the present invention thus constructedas previously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like to accurately detect theposition of an object located on the road even if the housing unit ismounted on the inaccurate position of the automotive vehicle.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordination system; a second housing position informationstoring step of storing second housing position information indicativeof the position of the housing unit in the second coordination system; afirst optical position information producing step of producing firstoptical position information indicative of the position of the opticalsection in the first coordination system on the basis of the imageinformation obtained by the imaging device in the first coordinationsystem; a first optical position information storing step of storing thefirst optical position information produced in the first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordination system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; a second optical position information storing step of storing thesecond optical position information produced in the second opticalposition information producing step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the basis of the imageinformation obtained by the imaging device in the second coordinationsystem. The camera calibrating program according to the presentinvention thus constructed as previously mentioned can calibrateparameters of the optical section mounted on an automotive vehicle orthe like to accurately detect the position of an object located on theroad.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: anoptical position information storing step of storing optical positioninformation indicative of the position of the optical section to apredetermined coordinate system; and a calibrating step of calibratingthe optical position information stored in the optical positioninformation storing step on the basis of the image information obtainedby the imaging device on the predetermined coordinate system. The cameracalibrating program according to the present invention thus constructedas previously mentioned can calibrate parameters of the optical sectionmounted on an automotive vehicle or the like to accurately detect theposition of an object located on the road.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordination system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of the position of the housingunit in the second coordination system in which a calibrating marker islocated; a first optical position information producing step ofproducing first optical position information indicative of the positionof the optical section in the first coordination system on the basis ofthe image information of the revising marker obtained by the imagingdevice; a first optical position information storing step of storing thefirst optical position information produced in the first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordination system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; an estimated location information producing step of producingestimated location information indicative of the position of thecalibrating marker to an image coordination system of the imaging deviceon the basis of the second optical position information produced in thesecond optical position information producing step; a second opticalposition information storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; an estimated location information storing step ofstoring the estimated location information produced in the estimatedlocation information estimating step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the basis of the imageinformation of the calibrating marker obtained by the imaging device andthe estimated location information stored in the estimated locationinformation storing step. The camera calibrating program according tothe present invention thus constructed as previously mentioned cancalibrate the second optical position information by using the simplecalibrating marker.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordination system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of the position of the housingunit in the second coordination system; a first optical positioninformation producing step of producing first optical positioninformation indicative of the position of the optical section in thefirst coordination system on the basis of the image information obtainedby the imaging device and indicative of the revising marker; a firstoptical position information storing step of storing the first opticalposition information produced in the first optical position informationproducing step; a second optical position information producing step ofproducing second optical position information indicative of the positionof the optical section in the second coordination system from the secondhousing position information stored in the second housing positioninformation storing step on the basis of the first housing positioninformation stored in the first housing position information storingstep and the first optical position information stored in the firstoptical position information storing step; a second optical positioninformation storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; and a calibrating step of calibrating the second opticalposition information stored in the second optical position informationstoring step on the basis of a motion vector of the image informationobtained by the imaging device in the second coordination system. Thecamera calibrating program according to the present invention thusconstructed as previously mentioned can calibrate the second opticalposition information by using the motion vector.

In accordance with the present invention, there is provided a cameracalibrating program of calibrating the position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough the optical section, and allowing a computer to execute: a firsthousing position information storing step of storing first housingposition information indicative of the position of the housing unit inthe first coordination system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of the position of the housingunit in the second coordination system in which an automotive vehicle islocated; a first optical position information producing step ofproducing first optical position information indicative of the positionof the optical section in the first coordination system on the basis ofthe image information of the revising marker obtained by the imagingdevice; a first optical position information storing step of storing thefirst optical position information produced in the first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of the position of the optical section in thesecond coordination system from the second housing position informationstored in the second housing position information storing step on thebasis of the first housing position information stored in the firsthousing position information storing step and the first optical positioninformation stored in the first optical position information storingstep; an estimated location information producing step of producingestimated location information indicative of the position of theautomotive vehicle to an image coordination system of the imaging deviceon the basis of the second optical position information produced in thesecond optical position information producing step; a second opticalposition information storing step of storing the second optical positioninformation produced in the second optical position informationproducing step; an estimated location information storing step ofstoring the estimated location information produced in the estimatedlocation information estimating step; and a calibrating step ofcalibrating the second optical position information stored in the secondoptical position information storing step on the optical section basisof the image information of the automotive vehicle obtained by theimaging device and the estimated location information stored in theestimated location information storing step. The camera calibratingprogram according to the present invention thus constructed aspreviously mentioned can calibrate the second optical positioninformation by using a portion of the automotive vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of a camera calibrating apparatus accordingto the present invention will be more clearly understood from thefollowing description taken in conjunction with the accompanyingdrawings:

FIG. 1 is a block diagram showing a camera calibrating apparatusaccording to the first embodiment of the present invention and a cameraexemplified as an imaging device;

FIG. 2 is a perspective view showing the first coordination system inwhich the camera shown in FIG. 1 is located;

FIG. 3 is a perspective view showing the second coordination system inwhich the camera shown in FIG. 1 is located;

FIG. 4 is a perspective view showing a coordination system of the camerashown in FIG. 1;

FIG. 5 is a plan view showing an image coordination system of the camerashown in FIG. 1;

FIG. 6 is a perspective view showing a parallel translation of thecamera shown in FIG. 1;

FIG. 7 is a perspective view showing a rotational motion of the camerashown in FIG. 1;

FIG. 8 is a side view showing a parallel translation of the camera shownin FIG. 1;

FIG. 9 is a side view showing a rotational motion of the camera shown inFIG. 1;

FIG. 10 is a block diagram showing an image location informationextracting section of the camera calibrating apparatus shown in FIG. 1;

FIG. 11 is a block diagram showing a computer and ECU to ensure thecamera calibrating apparatus shown in FIG. 1;

FIG. 12 is a block diagram showing a camera unit of the cameracalibrating apparatus shown in FIG. 1;

FIG. 13 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 1;

FIG. 14 is a block diagram showing a camera calibrating apparatusaccording to the second embodiment of the present invention and a cameraexemplified as an imaging device;

FIG. 15 a plan view showing an image coordination system of the camerashown in FIG. 14;

FIG. 16 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 14;

FIG. 17 is a block diagram showing the camera calibrating apparatusaccording to the third embodiment of the present invention and a cameraexemplified as an imaging device;

FIG. 18 is a perspective view showing the first coordination system inwhich the camera shown in FIG. 17 is located;

FIG. 19 is a perspective view showing the second coordination system inwhich the camera shown in FIG. 17 is located;

FIG. 20 is a perspective view showing a coordination system of thecamera shown in FIG. 17;

FIG. 21 is a side view showing an imaginary camera of the cameracalibrating apparatus shown in FIG. 17;

FIG. 22 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 23 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 24 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 25 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 17;

FIG. 26 is a block diagram showing a computer and ECU to ensure thecamera calibrating apparatus shown in FIG. 17;

FIG. 27 is a block diagram showing a camera unit of the cameracalibrating apparatus shown in FIG. 17;

FIG. 28 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 17;

FIG. 29 is a block diagram showing a camera calibrating apparatusaccording to the fourth embodiment of the present invention and a cameraexemplified as an image device;

FIG. 30 is a perspective view showing the first coordination system inwhich the camera shown in FIG. 29 is located;

FIG. 31 is a perspective view showing the second coordination system inwhich the camera shown in FIG. 29 is located;

FIG. 32 is a perspective view showing a coordination system of thecamera shown in FIG. 29;

FIG. 33 is a plan view showing an image coordination system of thecamera shown in FIG. 29;

FIG. 34 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 29;

FIG. 35 is a schematic view showing an operation of the calibratingsection of the camera calibrating apparatus shown in FIG. 29;

FIG. 36 is a block diagram showing a computer and ECU to ensure thecamera calibrating apparatus shown in FIG. 29;

FIG. 37 is a block diagram showing a camera unit of the cameracalibrating apparatus shown in FIG. 29;

FIG. 38 is a flow chart showing a calibrating operation of the cameracalibrating apparatus shown in FIG. 29;

FIG. 39 is a block diagram of a conventional camera calibratingapparatus and a camera exemplified as an imaging device;

FIG. 40 is a perspective view showing the first coordination system inwhich the camera shown in FIG. 39 is located; and

FIG. 41 is a perspective view showing the second coordination system inwhich the camera shown in FIG. 39 is located.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The embodiments of the camera calibrating apparatus according to thepresent invention will now be described in detail in accordance withaccompanying drawings.

(First Embodiment)

Referring now to the drawings, in particular to FIGS. 1 to 13, there isshown a first embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the first embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter. As shown in FIGS. 1 to 3, the camera calibrating apparatus100 is electrically connected to a camera 110 exemplified as an imagingdevice. The camera 110 includes a housing unit 111 and an opticalsection 112 supported by the housing unit 111 to obtain imageinformation through the optical section 112.

The camera calibrating apparatus 100 comprises a first housing positioninformation storing section 115 for storing first housing positioninformation indicative of the position of the housing unit 111 in thefirst coordination system 101, a second housing position informationstoring section 116 for storing second housing position informationindicative of the position of the housing unit 111 in the secondcoordination system 102, a revising marker position information storingsection 125 for storing revising marker position information indicativeof the position of a revising marker 105 in the first coordinationsystem 101, and a calibrating marker position information storingsection 126 for storing calibrating marker position informationindicative of the position of a calibrating marker 106 in the secondcoordination system 102.

The first coordination system 101 is defined in a first working spacesuch as a camera production plant to have X₁-axis, Y₁-axis, and Z₁-axis,while the revising marker 105 laid out in the first working space causesthe camera calibrating apparatus 100 to revise the camera 110 located inthe first working space. The revising marker 105 is constituted by aplurality of marking objects which are respectively located inpredetermined positions on the first coordination system 101, and whichare within a viewing field of the camera 110 located in the firstworking space.

The second coordination system 102 is defined in a second working spacesuch as an automotive vehicle production plant to have X₂-axis, Y₂-axis,and Z₂-axis, while X₂-axis and Y₂-axis collectively define X₂-Y₂ planewhich is represented by a road surface 102 a on which an automotivevehicle 108 is located. The calibrating marker 106 is constituted by twoor more marking objects which are respectively located at predeterminedpositions of the road surface 102 a in the second coordination system102.

The camera calibrating apparatus 100 is adapted to revise the camera 110in the first working space. The camera 110 is located at a predeterminedposition on the first coordination system 101, while the first housingposition information storing section 115 is adapted to store firsthousing position information indicative of the position of the housingunit 111. Here, the term “the calibration of the camera 110” is intendedto indicate the calculation of the accurate position of the opticalsection 112 of the camera 110 mounted on the automotive vehicle 108.

The camera 110 revised by the camera calibrating apparatus 110 ismounted on the automotive vehicle 108 to be located at a predeterminedposition on the second coordination system 102 in the second workingspace, while the second housing position information storing section 116is adapted to store second housing position information indicative ofthe position of the housing unit 111. Here, the second housing positioninformation indicates an accurate position in which the housing unit 111of the camera 110 is located when the camera 110 is mounted on theautomotive vehicle 108.

The camera calibrating apparatus 100 further comprises a first opticalposition information producing section 117 for producing first opticalposition information indicative of the position of the optical section112 in the first coordination system 101, and a first optical positioninformation storing section 118 for storing the first optical positioninformation produced by the first optical position information producingsection 117.

The first optical position information producing section 117 is adaptedto calculate the position of the optical section 112 in the firstcoordination system 101 from the calibrating marker position informationstored in the calibrating marker position information storing section125 on the basis of the image information of the calibrating marker 105obtained by the camera 110. Here, the term “the position of the opticalsection 112” is intended to indicate an optical center of the opticalsection 112 and the position of an optical axis, and other opticalparameters. The calculation of the position of the optical section 112in the first coordination system 101 is performed on the basis of amethod disclosed in the document 1 (R. Tsai, A versatile cameracalibration technique for high-accuracy 3D machine vision metrologyusing off-the-shelf TV cameras and lenses, IEEE Journal of Robotics andAutomation, RA-3 (4): 323-344, 1987).

The camera calibrating apparatus 100 further comprises a second opticalposition information producing section 120 for producing second opticalinformation indicative of the position of the optical section 112 in thesecond coordination system 102, and a second optical positioninformation storing section 130 for storing the second optical positioninformation produced by the second optical position informationproducing section 120.

The second optical position information producing section 120 is adaptedto produce second optical position information indicative of theposition of the optical section 112 in the second coordination system102 from the second housing position information stored in the secondhousing position information storing section 116 on the basis of thefirst housing position information stored by the first housing positioninformation storing section 115 and the first optical positioninformation stored in the first optical position information storingsection 118.

The second optical position information producing section 120 is adaptedto calculate the position of the optical section 112 in the secondcoordination system 102 on the basis of a following method.

The method is of comparing the position of the housing unit 111 to thefirst coordination system 101 with the position of the optical section112 in the first coordination system 101 to obtain positionalinformation between the housing unit 111 and the optical section 112,and to calculate information indicative of the position of the opticalsection 112 in the second coordination system 102 from the position ofthe housing unit 111 in the second coordination system 102 on the basisof the positional information between the housing unit 111 and theoptical section 112. Consequently, the second optical positioninformation indicates the position of the optical section 112 under thecondition that the housing unit 111 is located at an accurate positionof the automotive vehicle 108.

As shown in FIG. 4, the camera 110 mounted on the automotive vehicle 108in the second working space defines a camera coordination system 113having x-axis, y-axis, and z-axis on the basis of the second opticalposition information. The origin point of the camera coordination system113 is in register with the optical center of the optical section 112.The x-axis of the camera coordination system 113 extends in a horizontaldirection of the camera 110. The y-axis of the camera coordinationsystem 113 extends in a vertical direction of the camera 110. The z-axisof the camera coordination system 113 is in axial alignment with theoptical axis of the optical section 112.

The image coordination system 114 is defined on a plane spaced apartfrom the origin point of the camera coordination system 113 with a focallength “f” in the direction of the z-axis of the camera coordinationsystem 113. The image coordination system 114 has p-axis and q-axis. Thecamera 110 is adapted to obtain, as image information, an optical imagefocused on the image coordination system 114 through the optical section112.

The camera calibrating apparatus 100 further comprises an estimatedlocation information producing section 140 for producing estimatedlocation information of the calibrating marker 106 to the imagecoordination system 114 of the camera 110, and an estimated locationinformation storing section 150 for storing the estimated locationinformation produced by the estimated location information producingsection 140.

The estimated location information producing section 140 is adapted toproduce estimated location information of the calibrating marker 106 tothe image coordination system 114 of the camera 110 from the positioninformation of the calibrating marker 106 stored in the calibratingmarker position information storing section 126 on the basis of thesecond optical position information produced by the second opticalposition information producing section 120. The calculation of theestimated location of the calibrating marker 106 to the imagecoordination system 114 of the camera 110 is performed on the basis ofthe above mentioned document 1.

As shown in FIG. 5, the calibrating marker 106 located in the secondcoordination system 102 is focused on the image location Pn′ (n =1, 2,3, 4, 5, 6) of the image coordination system 114 through the opticalsection 112. Here, the image location Pn′ is in register with theestimated image location Pn (n=1, 2, 3, 4, 5, 6) calculated by theestimated image location calculating section 140 under the conditionthat the housing unit 111 is located at an accurate position of theautomotive vehicle 108, more specifically, the housing unit 111 islocated at the position represented by the second housing positioninformation with no deviation to the accurate position. However, thehousing unit 111, in general, is located at an inaccurate position withdeviation to the accurate position. This leads to the fact that thesecond optical position information is produced with deviation to theaccurate position. As a result, the estimated location Pn is spacedapart from the image location Pn′ of the image coordination system 114.

The second optical position information includes six parametersindicative of a parallel displacement and a rotational motion of thecamera coordination system 113 to the second coordination system 102.The six parameters are constituted by the parallel displacementcomponents of the X₂-axis, the Y₂-axis, and the Z₂-axis shown in FIG. 6and the rotational motion of the x-axis, the y-axis, and the z-axisshown in FIG. 7. The deviation of the second optical positioninformation includes components of the parallel displacement androtational motion. The housing unit 111 is, in general, located on theautomotive vehicle with the parallel displacement which is within therange of a few centimeters, and the rotational motion which is withinthe range of a few degrees.

The following description will be directed to the case that the imageinformation of the road surface 102 a obtained by the camera 110 isdisplayed in conjunction with an additional line when the automotivevehicle 108 is operated by an operator. Here, the housing unit 111 islocated at the position of the automotive vehicle 108 which is spacedapart from the road surface 302 a, while the additional line is locatedat the position spaced apart from a rear portion of the automotivevehicle 108. The height of the housing unit 111 to the road surface 302a is equal to 1,000 millimeters, while the distance of the additionalline to the rear portion is equal to 3000 millimeters.

When the housing unit 111 is located at the position spaced apart from adesign position of the automotive vehicle 108 in the Y-axis direction,more specifically, the parallel shift displacement of the housing unit111 to the design position is equal to 50 millimeters as shown in FIG.8, the displacement of the additional line is equal to 50 millimeters inY₂-axis direction. In this case, the elongation value of the imagelocation Pn′ to the estimated location Pn on the image coordinationsystem 114 shown in FIG. 5 can be negligible.

When, on the other hand, the housing unit 111 is mounted on theautomotive vehicle 108 with a rotational displacement of the housingunit 111 to the X₂-axis direction, more specifically, the rotationaldisplacement of the housing unit 111 to the X₂-axis direction is equalto one degree as shown in FIG. 9, the additional line is located with adisplacement of 184 millimeters in the Y₂-axis direction. In this case,the elongation value of the image location Pn′ to the estimated locationPn on the image coordination system 114 shown in FIG. 5 cannot benegligible.

As will be seen from the above description, the second optical positioninformation may be calibrated in consideration with only the rotationaldisplacement of the optical section 112.

The camera calibrating apparatus 100 further comprises a calibratingsection 160 for calibrating the second optical position informationstored in the second optical position information storing section 130 toorder to calibrate the above mentioned deviation of the second opticalposition information.

The calibrating section 160 is adapted to calibrate the second opticalposition information stored in the second optical position informationstoring section 130 on the basis of the image information of thecalibrating marker 106 obtained by the camera 110 and the estimatedlocation information stored in the estimated location informationstoring section 150.

The calibrating section 160 includes an image location informationextracting section 170 for extracting image location informationindicative of an image location of the calibrating marker 106 to theimage coordination system 114 of the camera 110 on the basis of theimage information of the calibrating marker 106 obtained by the camera110, a calibration value calculating section 180 for calculating acalibration value of the second optical position information stored inthe second optical position information storing section 130 on the basisof the image location information extracted by the image locationinformation extracting section 170 and the estimated locationinformation stored in the estimated location information storing section150, and an optical position information calibrating section 190 forcalibrating the second optical position information stored in the secondoptical position information storing section 130 on the basis of thecalibration value calculated by the calibration value calculatingsection 180.

The image location information extracting section 170 includes an imagedisplaying section 171 for displaying an image of the calibrating marker106 obtained by the camera 110, and an image location specifying section172 for specifying the image location Pn′ of the calibrating marker 106in the image of the calibrating marker 106 displayed by the imagedisplaying means 171 in order to extract the image location informationfrom the image information.

As shown in FIG. 10, the image displaying section 171 is adapted todisplay both the image of the calibrating marker 106 obtained by thecamera 110 and a cursor 174 for specifying the image location Pn′ of thecalibrating marker 106.

The image location specifying section 172 includes an up-arrow key 175a, a down-arrow key 175 b, a left-arrow key 175 c, and a right-arrow key175 d, all of which are capable of allowing the cursor 174 displayed onthe image information displaying section 171 to be moved in therespective directions, an enter key 176 for deciding the position of thecursor 174 as the image location Pn′ of the calibrating marker 106, anda number displaying section 177 for displaying number “n” of the markingobject of the calibrating marker 106, and an increment key 178 a and adecrement key 178 b each for changing the number “n” displayed on thenumber displaying section 177.

The calibration value calculating section 180 is adapted to calculatethe calibration value of the second optical position information on thebasis of the following method.

The following direction will be directed to the case that the deviationof the rotational component of the second optical position informationis considered as an overall deviation of the second optical positioninformation. $\begin{matrix}{J = {\sum\limits_{n = 1}^{N}\begin{bmatrix}{\left( {{p_{n}^{\prime}\left( {{R_{13}p_{n}} + {R_{23}q_{n}} + {R_{33}f}} \right)} - {f\left( {{R_{11}p} + {R_{21}q} + {R_{31}f}} \right)}} \right)^{2} +} \\\left( {{q_{n}^{\prime}\left( {{R_{13}p} + {R_{23}q_{n}} + {R_{33}f}} \right)} - {f\left( {{R_{12}p} + {R_{22}q} + {R_{32}f}} \right)}} \right)^{2}\end{bmatrix}}} & (1)\end{matrix}$

(p_(n)′, q_(n)′) and (p_(n), q_(n)) are linked by the following formula(2) on the basis of the formula (1). $\begin{matrix}\begin{matrix}{p^{\prime} = {f\frac{{R_{11}p} + {R_{21}q} + {R_{31}f}}{{R_{13}p} + {R_{23}q} + {R_{33}f}}}} \\{q^{\prime} = {f\frac{{R_{12}p} + {R_{22}q} + {R_{32}f}}{{R_{13}p} + {R_{23}q} + {R_{33}f}}}}\end{matrix} & (2)\end{matrix}$

R₁₁ to R₃₃ and θ, φ, φ are linked by the following formulas (3), (4),and (5). $\begin{matrix}\begin{matrix}{{Rx} = \begin{bmatrix}1 & 0 & 0 \\0 & {\cos\quad\theta} & {\sin\quad\theta} \\0 & {{- \sin}\quad\theta} & {\cos\quad\theta}\end{bmatrix}} \\{{Ry} = \begin{bmatrix}{\cos\quad\phi} & 0 & {{- \sin}\quad\phi} \\0 & 1 & 0 \\{\sin\quad\phi} & 0 & {\cos\quad\phi}\end{bmatrix}} \\{{Rz} = \begin{bmatrix}{\cos\quad\varphi} & {\sin\quad\varphi} & 0 \\{{- \sin}\quad\varphi} & {\cos\quad\varphi} & 0 \\0 & 0 & 1\end{bmatrix}}\end{matrix} & (3) \\{R = {{Rx} \times {Ry} \times {Rz}}} & (4) \\{R = \begin{bmatrix}R_{11} & R_{12} & R_{13} \\R_{21} & R_{22} & R_{23} \\R_{31} & R_{32} & R_{33}\end{bmatrix}} & (5)\end{matrix}$

In this embodiment, the calibrating marker 106 is constituted by sixmarking objects, however, the calibrating marker 106 may be constitutedby two or more marking objects. On the other hand, the calibration valuecalculating section 180 can calculate θ, φ, and φ if the calibratingmarker 106 is constituted by two marking objects. Additionally, thecalibration value calculating section 180 can calculate θ, φ, and φ at arelatively high accuracy if the calibrating marker 106 is constituted bythree or more marking objects.

As shown in FIG. 11, the camera calibrating apparatus 100 thusconstructed as previously mentioned can be realized by comprising thecomputer 191 for calibrating the camera 110, and the ECU 192 constitutedas an image controlling apparatus for controlling the camera 110.

The computer 191 is shown in FIG. 11(a) as including CPU, RAM, ROM, aninput/output interface, and other elements. The computer 191 iselectrically connected to the camera 110 at the first working space. Inthis embodiment, the computer 191 constitutes each of the first housingposition information storing section 115, the second housing positioninformation storing section 116, the first optical position informationproducing section 117, the first optical position information storingsection 118, the second optical position information producing section120, the calibrating marker position information storing section 125,the calibrating marker position information storing section 126, thesecond optical position storing section 130, the estimated locationinformation producing section 140, and the estimated locationinformation storing section 150.

The ECU 192 is shown in FIG. 11(c) as including CPU, RAM, ROM, aninput/output interface, and other elements. The ECU192 is mounted on theautomotive vehicle 108 to be electrically connected to the camera 110 inthe second working space. In this embodiment, the ECU 192 constituteseach of the second optical position information storing section 130, theestimated location information storing section 150, and the calibratingsection 160.

As will be seen from FIG. 11(b), the camera 110 is transferred to thesecond working space from the first working space with a recordablemedium 193 such as for example CD-ROM and a magnetic disk. Therecordable medium 193 has stored therein each of the second opticalposition information and the estimated location information to ensurethat the each of the second optical position information and theestimated location information is effectively transferred to the ECU 192from the computer 191.

In this embodiment, the camera 110 and the recordable medium 193 aretransferred to the second working space from the first working space,however, the camera 110, the second optical position information storingsection 130, the estimated location information storing section 150, andthe calibrating section 160 may be collectively transferred as a cameraassembly 194 to the second working space from the first working space aswill be seen from FIG. 12.

The following description will be directed to the operation of the firstembodiment of the camera calibrating apparatus according to the presentinvention. The second optical position information is calibrated by thecamera calibrating apparatus 100 through the steps shown in the FIG. 13.

The camera 110 is firstly located at the predetermined position on thefirst coordination system 101 in the first working space (in the stepS101). The first housing position information, the second housingposition information, the revising marker position information, and thecalibrating marker position information are then stored in the firsthousing position information storing section 115, the second housingposition information storing section 116, the revising marker positioninformation storing section 125, and the calibrating marker positioninformation storing section 126 (in the step S 102), respectively. Here,the first housing position information, the second housing positioninformation, the revising marker position information, and thecalibrating marker position information are obtained on the basis ofpositions predetermined in the design process and measured by ameasuring instrument.

The image information of the revising marker 105 is then obtained by thecamera 110 (in the step S103). The first optical position information isthen produced by the first optical position information producingsection 117 from the revising marker position information stored in therevising marker position information storing section 125 on the basis ofthe image information of the revising marker 105 obtained by the camera110 (in the step S104). The first optical position information producedby the first optical position information producing section 117 is thenstored in the first optical position information storing section 118 (inthe step S105).

The second optical position information is then produced by the secondoptical position information producing section 120 from the secondhousing position information stored in the second housing positioninformation storing section 116 on the basis of the first housingposition information stored in the first housing position informationstoring section 115 and the first optical position information stored inthe first optical position information storing section 118 (in the stepS106). The second optical position information produced by the secondoptical position information producing section 120 is then stored in thesecond optical position information storing section 130 (in the stepS107).

The estimated location information indicative of the estimated locationof the calibrating marker 106 is then produced by the estimated locationinformation producing section 140 from the calibrating marker positioninformation stored in the calibrating marker position informationstoring section 126 on the basis of the second optical positioninformation produced by the second optical position informationproducing section 120 (in the step S108). The estimated locationinformation produced by the estimated location information producingsection 140 is stored in the estimated location information storingsection 150 (in the step S109).

The camera 110 and the recordable medium 193 are then transferred to thesecond working space from the first working space. The camera 110 isthen mounted to the automotive vehicle 108 to be located at thepredetermined position on the second coordination system 102 (in thestep S110).

The image information of the calibrating marker 106 is then obtained bythe camera 110 (in the step S111). The image information of thecalibrating marker 106 obtained by the camera 110 is then displayed onthe image displaying section 171 (in the step S112) as shown in FIG. 10.The image location Pn′ of the calibrating marker is specified by theimage location specifying section 172 (in the step S113). Here, theincrement key 178 a and the decrement key 178 b are operated by anoperator, while the number “n” of the marking object of the calibratingmarker 106 displayed by the number displaying section 177 in response tothe increment key 178 a and the decrement key 178 b. The up-arrow key175 a, the down-arrow key 175 b, the left-arrow key 175 c, and theright-arrow key 175 d are operated by the operator, while the cursor 174displayed on the image displaying section 171 is moved in response toeach of the up-arrow key 175 a, the down-arrow key 175 b, the left-arrowkey 175 c, and the right-arrow key 175 d. The enter key 176 is operatedby the operator, while the position of the cursor 176 is decided as theimage location Pn′ of the number “n” of the calibrating marker 106displayed on the number displaying section 177.

The calibration value of the second optical position information storedin the second optical position information storing section 130 is thencalculated by the calibration value calculating section 180 on the basisof the image location information extracted by the image locationinformation extracting section 170 and the estimated locationinformation stored in the estimated location information storing section150 (in the step S114).

The second optical position information stored in the second opticalposition information storing section 130 is finally calibrated by theoptical position information calibrating section 190 on the basis of thecalibration value calculated by the calibration value calculatingsection 180 (in the step S115). In this embodiment, the above mentionedsteps 101 to 115 are executed as a program by a computer.

From the above detail description, it will be understood that thepresent embodiment of the camera calibrating apparatus 100 according tothe present invention can calibrate the optical parameters of the camera110 mounted on the automotive vehicle 108 to ensure an accurate positionof an object located on the road is detected.

In this embodiment, the camera calibrating apparatus 100 according tothe present invention can calibrate the second optical position by usingthe simple calibrating marker 106.

In this embodiment, the camera calibrating apparatus 100 according tothe present invention can specify the image location of the calibratingmarker 106 to ensure that the image location information indicative ofthe image location of the calibrating marker 106 is extracted.

(Second Embodiment)

Referring now to the drawings, in particular to FIGS. 14 to 16, there isshown a second embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the second embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter. The constitutional elements of the second embodiment of thecamera calibrating apparatus the same as those of the first embodimentof the camera calibrating apparatus will not be described but bear thesame reference numerals and legends as those of the first embodiment ofthe camera calibrating apparatus.

In FIG. 14, the camera calibrating apparatus 200 comprises a calibratingsection 260 for calibrating the second optical position informationstored in the second optical position information storing section 130.

The calibrating section 260 is adapted to calibrate the second opticalposition information stored in the second optical position informationstoring section 130 on the basis of the image information of thecalibrating marker 106 obtained by the camera 110 and the estimatedlocation information stored in the estimated location informationstoring section 150.

The calibrating section 260 includes an image location informationextracting section 270 for extracting image location informationindicative of an image location of the calibrating marker 106 to theimage coordination system 114 of the camera 110 on the basis of theimage information of the calibrating marker 106 obtained by the camera110, a calibration value calculating section 280 for calculating acalibration value to the second optical position information stored inthe second optical position information storing section 130 on the basisof the image location information extracted by the image locationinformation extracting section 270 and the estimated locationinformation stored in the estimated location information storing section150, and an optical position information calibrating section 290 forcalibrating the second optical position information stored in the secondoptical position information storing section 130 on the basis of thecalibration value calculated by the calibration value calculatingsection 280.

The image location information extracting section 270 includes anestimated area information storing section 271 for storing estimatedarea information (see FIG. 15) indicative of an estimated area 274 ofthe calibrating marker 106 to the image coordination system 114 of thecamera 110, and an image location searching section 272 for searchingthe image location Pn′ of the calibrating marker 106 from the imageinformation of the calibrating marker 106 obtained by the camera 110 onthe basis of the estimated area information stored by the estimated areainformation storing section 271 and the estimated location informationstored by the estimated location information storing section 150 toensure that the image location information is extracted.

The calibration value calculating section 280 is adapted to calculatethe calibration value to the second optical position information on thebasis of a method which is the same as the method of the calibrationvalue calculating section 180 of the first embodiment.

The following description will be directed to the operation of thesecond embodiment of the camera calibrating apparatus according to thepresent invention. The second optical position information is calibratedby the camera calibrating apparatus 200 through the following stepsshown in FIG. 16.

The camera 110 is firstly located at the predetermined position on thefirst coordination system 101 in a first working space (in the stepS201). The first housing position information, the second housingposition information, the revising marker position information, and thecalibrating marker position information are then stored in the firsthousing position information storing section 115, the second housingposition information storing section 116, the revising marker positioninformation storing section 125, and the calibrating marker positioninformation storing section 126 (in the step S202), respectively. Here,the first housing position information, the second housing positioninformation, the revising marker position information, and thecalibrating marker position information are obtained on the basis ofpositions predetermined in the design process and measured by ameasuring instrument.

The image information of the revising marker 105 is then obtained by thecamera 110 (in the step S203). The first optical position information isthen produced by the first optical position information producingsection 117 from the revising marker position information stored in therevising marker position information storing section 125 on the basis ofthe image information of the revising marker 105 obtained by the camera110 (in the step S204). The first optical position information producedby the first optical position information producing section 117 is thenstored in the first optical position information storing section 118 (inthe step S205).

The second optical position information is then produced by the secondoptical position information producing section 120 from the secondhousing position information stored in the second housing positioninformation storing section 116 on the basis of the first housingposition information stored in the first housing position informationstoring section 115 and the first optical position information stored inthe first optical position information storing section 118 (in the stepS206). The second optical position information produced by the secondoptical position information producing section 120 is then stored in thesecond optical position information storing section 130 (in the stepS207).

The estimated location information indicative of the position of thecalibrating marker 106 to the image coordination system of the camera110 is then produced by the estimated location information producingsection 140 from the calibrating marker position information stored inthe calibrating marker position information storing section 126 on thebasis of the second optical position information produced by the secondoptical position information producing section 120 (in the step S208).The estimated location information produced by the estimated locationinformation producing section 140 is stored in the estimated locationinformation storing section 150 (in the step S209).

The camera 110 and the recordable medium 193 are then transferred to thesecond working space from the first working space. The camera 110 isthen mounted to the automotive vehicle 108 to be located at thepredetermined position on the second coordination system 102 (in thestep S210).

The image information of the calibrating marker 106 is then obtained bythe camera 110 (in the step S211). As shown in FIG. 15, the imagelocation Pn′ is searched by the image location searching section 272 inthe estimated area 274 at the estimated location Pn of the calibratingmarker 106. The image location information is then extracted by theimage location information extracting section 270.

The calibration value to the second optical position information storedin the second optical position information storing section 130 is thencalculated by the calibration value calculating section 280 on the basisof the image location information extracted by the image locationinformation extracting section 270 and the estimated locationinformation stored in the estimated location information storing section150 (in the step S213).

The second optical position information stored in the second opticalposition information storing section 130 is finally calibrated by theoptical position information calibrating section 290 on the basis of thecalibration value calculated by the calibration value calculatingsection 280 (in the step S214). In this embodiment, the above mentionedsteps 101 to 115 may be executed as a program by a computer.

From the above detailed description, it will be understood that thesecond embodiment of the camera calibrating apparatus 200 can easilyextract the image location information of the calibrating marker 106 byreason that the image location of the calibrating marker 106 is searchedby the image location searching section 272.

(Third Embodiment)

Referring now to the drawings, in particular to FIGS. 17 to 28, there isshown a third embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the third embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter.

As shown in FIGS. 17 to 19, the camera calibrating apparatus 300 iselectrically connected to a camera 310 as an imaging device. The camera310 includes a housing unit 311 and an optical section 312 supported bythe housing unit 311 to obtain image information through the opticalsection 312.

The camera calibrating apparatus 300 comprises a first housing positioninformation storing section 315 for storing first housing positioninformation indicative of the position of the housing unit 311 to thefirst coordination system 301, a second housing position informationstoring section 316 for storing second housing position informationindicative of the position of the housing unit 311 in the secondcoordination system 302, and a revising marker position informationstoring section 325 for storing revising marker position informationindicative of the position of a revising marker 305 in the firstcoordination system 301.

The first coordination system 301 is defined in a first working spacesuch as a camera production plant to have X₁-axis, Y₁-axis, and Z₁-axis,while the revising marker 305 laid out in the first working space causesthe camera calibrating apparatus 300 to revise the camera 310 located inthe first working space. The revising marker 305 is constituted by aplurality of marking objects which are respectively located atpredetermined positions on the first coordination system 301, and whichare within a viewing field of the camera 310 located in the firstworking space.

The second coordination system 302 is defined in a second working spacesuch as an automotive vehicle production plant. The second coordinationsystem 302 has X₂-axis, Y₂-axis, and Z₂-axis. The X₂-axis and theY₂-axis collectively define X₂-Y₂ plane to be represented by a roadsurface 302 a on which an automotive vehicle 308 is located. Thedividing marker 307 is located on a bumper 309 of the automotive vehicle308. The dividing marker 307 is constituted by two marking objects whichis within a viewing field of the camera 310 mounted on the automotivevehicle 308. The dividing marker 307 is located directly below thecamera 310 mounted on the automotive vehicle 308.

The camera calibrating apparatus 300 is adapted to revise the camera 310in the first working space. The camera 310 is located at a predeterminedposition on the first coordination system 301, while the first housingposition information storing section 315 is adapted to store firsthousing position information indicative of the position of the housingunit 311. Here, the term “the calibration of the camera 310” is intendedto indicate the calculation of the accurate position of the opticalsection 312 of the camera 310 mounted on the automotive vehicle 308.

The camera 310 revised by the camera calibrating apparatus 310 ismounted on the automotive vehicle 308 to be located at a predeterminedposition on the second coordination system 302 in the second workingspace, while the second housing position information storing section 316is adapted to store second housing position information indicative ofthe position of the housing unit 311. Here, the second housing positioninformation indicates an accurate position in which the housing unit 311of the camera 310 is located when the camera 310 is mounted on theautomotive vehicle 308.

The dividing marker 307 is located on the bumper 309 of the automotivevehicle 308 in spaced relationship with the position of the housing unit311 represented by the second housing position information. This meansthat the dividing marker 307 is not moved with respect to the housingunit 311 when the automotive vehicle 308 is traveling on the road 302 a.

The camera calibrating apparatus 300 further comprises a first opticalposition information producing section 317 for producing first opticalposition information indicative of the position of the optical section312 to the first coordination system 301, and a first optical positioninformation storing section 318 for storing the first optical positioninformation produced by the first optical position information producingsection 317.

The first optical position information producing section 317 is adaptedto calculate the position of the optical section 312 to the firstcoordination system 301 from the revising marker position informationstored in the revising marker position information storing section 325on the basis of the image information of the revising marker 305obtained by the camera 310. Here, the term “the position of the opticalsection 312” is intended to indicate an optical center of the opticalsection 312 and the position of an optical axis, and other opticalparameters. The calculation of the position of the optical section 312to the first coordination system 301 is performed on the basis of themethod disclosed in the document 1.

The camera calibrating apparatus 300 further comprises a second opticalposition information producing section 320 for producing second opticalinformation indicative of the position of the optical section 312 in thesecond coordination system 302, and a second optical positioninformation storing section 330 for storing the second optical positioninformation produced by the second optical position informationproducing section 320.

The second optical position information producing section 320 is adaptedto produce second optical position information indicative of theposition of the optical section 312 in the second coordination system302 from the second housing position information stored in the secondhousing position information storing section 316 on the basis of thefirst housing position information stored by the first housing positioninformation storing section 315 and the first optical positioninformation stored in the first optical position information storingsection 318.

The second optical position information producing section 320 is adaptedto calculate the position of the optical section 312 in the secondcoordination system 302 on the basis of a method which is the same asthe method of the second optical position information producing section120 of the first embodiment.

As shown in FIG. 20, the camera 310 mounted on the automotive vehicle308 located in the second working space defines a camera coordinationsystem 313 having x-axis, y-axis, and z-axis on the basis of the secondoptical position information. The origin point of the cameracoordination system 313 is in register with the optical center of theoptical section 312. The x-axis of the camera coordination system 313extends in a horizontal direction of the camera 310. The y-axis of thecamera coordination system 313 extends in a vertical direction of thecamera 310. The z-axis of the camera coordination system 313 is in axialalignment with the optical axis of the optical section 312.

The image coordination system 314 is defined on a plane spaced apartfrom the origin point of the camera coordination system 313 with a focallength “f” in the direction of the z-axis of the camera coordinationsystem 313. The image coordination system 314 has p-axis and q-axis. Thecamera 310 is adapted to obtain, as image information, an optical imagefocused on the image coordination system 314 through the optical section312.

The camera calibrating apparatus 300 further comprises a calibratingsection 360 for calibrating the second optical position informationstored in the second optical position information storing section 330 onthe basis of the motion vector of the image information obtained by thecamera 310 in the second coordination system 302.

The calibrating section 360 includes a plane-projected image producingsection 361 for producing a plane-projected image from the imageinformation obtained by the camera 310 on the second coordination system302, a plane-projected image dividing section 362 for dividing theplane-projected image produced by the plane-projected image producingsection 361 into a plurality of image segments, a motion vectorextracting section 363 for extracting the motion vector from the imagesegments divided by the plane-projected image dividing section362, acalibration value calculating section 364 for calculating a calibrationvalue of the second optical position information stored by the secondoptical position information storing section 330 on the basis of themotion vector extracted by the motion vector extracting section 363, andan optical position information calibrating section 365 for calibratingthe second optical position information stored by the second opticalposition information storing section 330 on the basis of the calibrationvalue calculated by the calibration value calculating section 364.

As shown in FIG. 21, the plane-projected image produced by theplane-projected image producing section 361 is the sama as an imagetaken by an imaginary camera 370 located on a plane which defines animage coordination system 371 parallel to the road surface 302 a. Theimage of the road surface 302 a focused on the image coordination system371 is the same as the reduced-size road surface 302 a.

The plane-projected image producing section 361 is adapted to produceplane-projected image on the basis of a following method.

In FIG. 20, the angle of the x-axis of the image coordination system 313to the road surface 302 a is represented by a legend α. The angle of they-axis of the image coordination system 313 to the road surface 302 a isrepresented by a legend β. The distance between the origin point of thecamera coordination system 313 and an intersecting point of theextension line of the z-axis of the camera coordination system 313 tothe road surface 302 a is represented by a legend “c”. The X₂-Y₂ surfaceof the second coordination system 302, i.e., the road surface 302 a isrepresented by following equations (6) and (7). $\begin{matrix}\left\{ \begin{matrix}{a = {\sin\quad\alpha}} \\{b = {\sin\quad\beta}}\end{matrix} \right. & (6)\end{matrix}$z=ax+by+c   (7)

The transformation to the camera coordination system 313 from the imagecoordination system 314 is represented by a following equation (8).Here, the legend (p, q) represents ordinate and abscissa of the imagelocation P′ to the image coordination system 314. $\begin{matrix}\left\{ \begin{matrix}{x = \frac{cp}{f - {ap} - {bp}}} \\{y = \frac{cq}{f - {ap} - {bp}}} \\{z = \frac{cf}{f - {ap} - {bp}}}\end{matrix} \right. & (8) \\{\begin{bmatrix}X \\Y \\Z\end{bmatrix} = {{\begin{bmatrix}R_{11} & R_{12} & R_{13} \\R_{21} & R_{22} & R_{23} \\R_{31} & R_{32} & R_{33}\end{bmatrix}\begin{bmatrix}x \\y \\z\end{bmatrix}} + \begin{bmatrix}T_{x} \\T_{y} \\T_{z}\end{bmatrix}}} & (9)\end{matrix}$

In the equation (9), the translation vector “T” indicates a direction ofthe origin point of the second coordination system 302 to the originpoint of the camera coordination system 313 and a distance between theorigin point of the second coordination system 302 and the origin pointof the camera coordination system 313. The rotation matrix “R”represents a deviation between the second coordination system 302 andthe camera coordination system 313 in the rotational direction. When thehousing unit 311 is mounted on the slanted position on the automotivevehicle 308, the rotation matrix “R” is represented by the equations (3)to (5) shown in the first embodiment. Here, the legends θ, φ, and φrepresent the angle of the camera coordination system 313 to the x-axisof the camera coordination system 313, the angle of the cameracoordination system 313 to the y-axis of the camera coordination system313, and the angle of the camera coordination system 313 to the z-axisof the camera coordination system 313, respectively.

When the housing unit 311 is mounted on the accurate position on theautomotive vehicle 308, the plane-projected image producing section 361is adapted to produce a plane-projected image shown in FIG. 22(b) fromthe image information shown in FIG. 22(a). In this plane-projectedimage, the lines 373 and 374 are in parallel relationship with eachother by reason that the plane-projected image is the same as the imagetaken by the imaginary camera 370 located in parallel relationship withthe road surface 302 a.

When the camera coordination system 313 is moved around the x-axis underthe condition that the housing unit 311 is downwardly slanted withrespect to the automotive vehicle 308, the plane-projected imageproducing section 361 is adapted to produce a plane-projected imageshown in FIG. 23(b) from the image information shown in FIG. 23(a).

When the camera coordination system 313 is moved around the y-axis underthe condition that the housing unit 311 is leftwardly panned withrespect to the automotive vehicle 308, the plane-projected imageproducing section 361 is adapted to produce a plane-projected imageshown in FIG. 24(b) from the image information shown in FIG. 24(a).

When the camera coordination system 313 is moved around the z-axis underthe condition that the housing unit 311 is leftwardly slanted withrespect to the automotive vehicle 308, the plane-projected imageproducing section 361 is adapted to produce a plane-projected imageshown in FIG. 25(b) from the image information shown in FIG. 25(a).

As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the plane-projectedimage dividing section 362 is adapted to divide the plane-projectedimage into a plurality of image segments on the basis of the imageinformation of the dividing markers 307 obtained by the camera 310.

The plane-projected image dividing section 362 is adapted to superimposedividing lines 383 and 384 on the plane-projected image on the basis ofa base line 380 passing through the dividing markers 307. The dividingline 383 is in perpendicular relationship with the base line 380, andpasses thought an intermediate point 381 defined between the dividingmarkers 307. On the other hand, the dividing line 384 is in parallel andspaced relationship with the base line 380.

When the housing unit 311 is mounted on the slanted position of theautomotive vehicle 308, more specifically, the camera coordinationsystem 313 is moved around the z-axis, the base line 380 has a tilt tothe right as will be seen from the plane-projected image shown in FIG.25(b).

As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the motion vectorextracting section 363 is adapted to extract the motion vectors 388 a,388 b, 388 c, and 388 d from the four image segments 386 a, 386 b, 386c, and 386 d divided by the plane-projected image dividing section 362.

The directions of the motion vectors 388 a to 388 d are equal to oneanother when the automotive vehicle 308 is linearly traveling by reasonthat the motion vectors 388 a to 388 d are calculated on the basis ofthe flow of the fragmental image of the image information obtained fromthe motion of the automotive vehicle 308, i.e., the motion of the camera310 to the road. The following description will be directed to themotion vectors 388 a to 388 d extracted from the image informationobtained under the automotive vehicle 308 is linearly traveling.

When the housing unit 311 is mounted on the accurate position of theautomotive vehicle 308, the amplitudes of the motion vectors 388 a, 388b, 388 c, and 388d are equal to one another as shown in FIG. 22(b).

When the housing unit 311 is mounted on the downwardly slanted positionof the automotive vehicle 308, more specifically, the cameracoordination system 313 is moved around the x-axis, the motion vectors388 a and 338 b of the upper side fragmental image is larger than themotion vectors 388 a and 338 b of the lower side fragmental image asshown in FIG. 23(b).

When the housing unit 311 is mounted on the leftwardly slanted positionof the automotive vehicle 308, more specifically, the cameracoordination system 313 is moved around the y-axis, the motion vectors388 b and 338 d of the right side fragmental image is larger than themotion vectors 388 a and 338 c of the left side fragmental image asshown in FIG. 24(b).

The calibration value calculating section 364 is adapted to produce acalibration value of the second optical position information on thebasis of a following method.

As shown in FIGS. 22(c), 23(c), and 24(c), the vector 390 a equal inamplitude to the motion vector 388 a, the vector 390 b equal inamplitude to the motion vector 388 b, the vector 390 c equal inamplitude to the motion vector 388 c, and the vector 390 d equal inamplitude to the motion vector 388 d are located on a composited vectorcalculating coordination system 391.

Here, the vector 390 a is located at the origin point of the compositedvector calculating coordination system 391 in the direction of anupper-leftward angle of 45 degrees. The vector 390 b is located at theorigin point of the composited vector calculating coordination system391 in the direction of an upper-rightward angle of 45 degrees. Thevector 390 c is located at the origin point of the composited vectorcalculating coordination system 391 in the direction of a lower-leftwardangle of 45 degrees. The vector 390 a is located at the origin point ofthe composited vector calculating coordination system 391 in thedirection of a lower-rightward angle of 45 degrees.

The composition of the vectors 390 a to 390 d is performed on thecomposited vector calculating coordination system 391.

When the housing unit 311 is mounted on the accurate position of theautomotive vehicle 308, the amplitude of the composited vector 392 isequal to zero by reason that the amplitudes of the motion vectors 388 a,388 b, 388 c, and 388 d are equal to one another as shown in FIG. 22(b).Here, each of the rotation angle θ to the x-axis, the rotation angle φto the y-axis, and the rotation angle φ to the z-axis is equal to zeroon the camera coordination system 313.

When the housing unit 311 is mounted on the downwardly slanted positionof the automotive vehicle 308, and more specifically, the cameracoordination system 313 is moved around the x-axis, the compositedvector 392 points upward by reason that the vectors 390 a and 390 b arerespectively larger in amplitude than the vectors 390a and 390 b asshown in FIG. 23(b). The rotation angle θ to the x-axis is calculated onthe basis of the equation (9) under the condition that the calculatedrotation matrix “R” leads to the fact that the composited vector 392 isequal to zero.

When the housing unit 311 is mounted on the leftwardly slanted positionof the automotive vehicle 308, and more specifically, the cameracoordination system 313 is moved around the y-axis, the compositedvector 392 points rightward by reason that the vectors 390 b and 390 dare respectively larger in amplitude than the vectors 390a and 390 c asshown in FIG. 24(b). The rotation angle φ to the y-axis is calculated onthe basis of the equation (9) under the condition that the calculatedrotation matrix “R” leads to the fact that the composited vector 392 isequal to zero.

When, on the other hand, the housing unit 311 is mounted on the slantedposition of the automotive vehicle 308, and more specifically, thecamera coordination system 313 is moved around the z-axis, the rotationangle φ to the z-axis of the camera coordination system 313 iscalculated on the basis of the slope of the base line 380 on the imagecoordination system 371 of the plane-projected image shown in FIG.25(b).

As shown in FIG. 26, the camera calibrating apparatus 300 thusconstructed as previously mentioned can be realized by comprising acomputer 391 for adjusting the camera 310, ECU 392 constituted as animage controlling apparatus for controlling the camera 310, and otherelements.

The computer 391 is shown in FIG. 26(a) as including CPU, RAM, ROM, aninput/output interface, and other units. The computer 391 iselectrically connected to the camera 310 at the first working space. Inthis embodiment, the computer 391 constitutes each of the first housingposition information storing section 315, the second housing positioninformation storing section 316, the first optical position informationproducing section 317, the first optical position information storingsection 318, the second optical position information producing section320, the revising marker position information storing section 325, andthe second optical position information storing section 330.

The ECU 392 is shown in FIG. 26(c) as including CPU, RAM, ROM, aninput/output interface, and other elements. The ECU 392 is mounted onthe automotive vehicle 308 to be electrically connected to the camera310 in the second working space. In this embodiment, the ECU 392constitutes each of the second optical position information storingsection 330 and the calibrating section 360.

As shown in FIG. 26(b), the camera 310 is transferred to the secondworking space from the first working space with a recordable medium 393such as for example CD-ROM and a magnetic disk. The recordable medium393 has stored therein the second optical position information to ensurethat the second optical position information is effectively transferredto the ECU 392 from the computer 391.

In this embodiment, the camera 310 and the recordable medium 393 aretransferred to the second working space from the first working space,however, the camera 310, the second optical position information storingsection 330, and the calibrating section 360 may be collectivelytransferred as a camera unit 194 to the second working space from thefirst working space as shown in FIG. 27.

The following description will be directed to the operation of the thirdembodiment of the camera calibrating apparatus according to the presentinvention.

The second optical position information is calibrated by the cameracalibrating apparatus 300 through the steps shown in the FIG. 28.

The camera 310 is firstly located at the predetermined position on thefirst coordination system 301 in the first working space (in the stepS301). The first housing position information, the second housingposition information, and the revising marker position information arethen stored in the first housing position information storing section315, the second housing position information storing section 316, andthe revising marker position information storing section 325,respectively (in the step S302). Here, the first housing positioninformation, the second housing position information, and the revisingmarker position information are obtained on the basis of positionspredetermined in the design process and measured by a measuringinstrument.

The image information of the revising marker 105 is then obtained by thecamera 310 (in the step S303). The first optical position information isthen produced by the first optical position information producingsection 317 from the revising marker position information stored in therevising marker position information storing section 325 on the basis ofthe image information of the revising marker 305 obtained by the camera310 (in the step S304). The first optical position information producedby the first optical position information producing section 317 is thenstored in the first optical position information storing section 318 (inthe step S305).

The second optical position information is then produced by the secondoptical position information producing section 320 from the secondhousing position information stored in the second housing positioninformation storing section 316 on the basis of the first housingposition information stored in the first housing position informationstoring section 315 and the first optical position information stored inthe first optical position information storing section 318 (in the stepS306). The second optical position information produced by the secondoptical position information producing section 320 is then stored in thesecond optical position information storing section 330 (in the stepS307).

The camera 310 and the recordable medium 393 are then transferred to thesecond working space from the first working space. The camera 310 isthen mounted to the automotive vehicle 308 to be located at apredetermined position on the second coordination system 302 (in thestep S308).

The image of the road surface 302 a is then taken by the camera 310 (inthe step S309). As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), theplane-projected image is produced from the image information obtained bythe camera 310 on the second coordination system 302 (in the step S310).

As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the plane-projectedimage produced by the plane-projected image producing section 361 isdivided into a plurality of image segments 386 a to 386 d by theplane-projected image dividing section 362 on the basis of the imageinformation of the dividing markers 307 obtained by the camera 310 (inthe step S311).

As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the motion vectors areextracted by the motion vector extracting section 363 from the imagesegments 386 a to 386 d divided by the plane-projected image dividingsection 362 (in the step S312).

As shown in FIGS. 25(b), the calibration value on z-axis of the secondoptical position information stored in the second optical positioninformation storing section 330 is then calculated by the calibrationvalue calculating section 365 on the basis of the slope of the base line380 on the image coordination system 371 of the plane-projected imageshown in FIG. 25(b) (in the step S313). As shown in FIGS. 22(c), 23(c),and 24(c), the calibration value on each of x-axis and y-axis of thesecond optical position information stored in the second opticalposition information storing section 330 is then calculated by thecalibration value calculating section 365 on the basis of the motionvectors extracted by the motion vector extracting section 363 (in thestep S314).

The second optical position information stored in the second opticalposition information storing section 330 is finally calibrated by theoptical position information calibrating section 365 on the basis of thecalibration value calculated by the calibration value calculatingsection 364 (in the step S315). In this embodiment, the above mentionedsteps 301 to 315 may be executed as a program by a computer.

From the above detailed description, it will be understood that thecamera calibrating apparatus 300 can calibrate the second opticalposition information by using the motion vectors 388 a to 388 d.

In this embodiment, the camera calibrating apparatus 300 can easilyextract the motion vectors from the respective image segments 386 a to386 d.

In this embodiment, the camera calibrating apparatus 300 can accuratelydivide the plane-projected image by using the dividing marker 307.

(Fourth Embodiment)

Referring now to the drawings, in particular to FIGS. 29 to 38, there isshown a fourth embodiment of a camera calibrating apparatus according tothe present invention.

The construction of the fourth embodiment of the camera calibratingapparatus according to the present invention will now be describedhereinafter.

As shown in FIGS. 29 to 31, the camera calibrating apparatus 400 iselectrically connected to a camera 410 as an imaging device. The camera410 includes a housing unit 411 and an optical section 412 supported bythe housing unit 411 to obtain image information through the opticalsection 412.

The camera calibrating apparatus 400 comprises a first housing positioninformation storing section 415 for storing first housing positioninformation indicative of the position of the housing unit 411 to thefirst coordination system 401, a second housing position informationstoring section 416 for storing second housing position informationindicative of the position of the housing unit 411 in the secondcoordination system 402, a revising marker position information storingsection 425 for storing revising marker position information indicativeof the position of a revising marker 405 to the first coordinationsystem 401, and a vehicle position information storing section 426 forstoring vehicle position information indicative of the position of, forexample, a rear bumper 409 of an automotive vehicle 408 in the secondcoordination system 402.

The first coordination system 401 is defined in a first working spacesuch as a camera production plant to have X₁-axis, Y₁-axis, and Z₁-axis.The revising marker 405 laid out in the first working space causes thecamera calibrating apparatus 400 is revise the camera 410 located in thefirst working space. The revising marker 405 is constituted by aplurality of marking objects which are respectively located atpredetermined positions on the first coordination system 401, and whichare within a viewing field of the camera 410 located in the firstworking space.

The second coordination system 402 is defined in a second working spacesuch as an automotive vehicle production plant to have X₂-axis, Y₂-axis,and Z₂-axis, while X₂-axis and Y₂-axis collectively define X₂-Y₂ planewhich is represented by a road surface 402 a on which an automotivevehicle 408 is located. The calibrating plate 406 painted in a singlecolor is located on the road surface 402 a. The monochromaticcalibrating plate 406 located directly below the automotive vehicle 408is different in brightness, chromaticity, and saturation of color fromthe automotive vehicle 408.

The camera calibrating apparatus 400 is adapted to revise the camera 410in the first working space. The camera 410 is located at a predeterminedposition on the first coordination system 401, while the first housingposition information storing section 415 is adapted to store firsthousing position information indicative of the position of the housingunit 411. Here, the term “the calibration of the camera 410” is intendedto indicate the calculation of the accurate position of the opticalsection 412 of the camera 410 mounted on the automotive vehicle 408.

The camera 410 revised by the camera calibrating apparatus 410 ismounted on the automotive vehicle 408 to be located at a predeterminedposition on the second coordination system 402 in the second workingspace, while the second housing position information storing section 416is adapted to store second housing position information indicative ofthe position of the housing unit 411. Here, the second housing positioninformation indicates an accurate position in which the housing unit 411of the camera 410 is located when the camera 410 is mounted on theautomotive vehicle 408.

The camera calibrating apparatus 400 further comprises a first opticalposition information producing section 417 for producing first opticalposition information indicative of the position of the optical section412 to the first coordination system 401, and a first optical positioninformation storing section 418 for storing the first optical positioninformation produced by the first optical position information producingsection 417.

The first optical position information producing section 417 is adaptedto calculate the position of the optical section 412 to the firstcoordination system 401 from the revising marker position informationstored in the revising marker position information storing section 425on the basis of the image information of the revising marker 305obtained by the camera 410. Here, the term “the position of the opticalsection 412” is intended to indicate an optical center of the opticalsection 412 and the position of an optical axis, and other opticalparameters. The calculation of the position of the optical section 412to the first coordination system 401 is performed on the basis of themethod disclosed in the document 1.

The camera calibrating apparatus 400 further comprises a second opticalposition information producing section 420 for producing second opticalinformation indicative of the position of the optical section 412 in thesecond coordination system 402, and a second optical positioninformation storing section 430 for storing the second optical positioninformation produced by the second optical position informationproducing section 420.

The second optical position information producing section 420 is adaptedto produce second optical position information indicative of theposition of the optical section 412 in the second coordination system402 from the second housing position information stored in the secondhousing position information storing section 416 on the basis of thefirst housing position information stored by the first housing positioninformation storing section 415 and the first optical positioninformation stored in the first optical position information storingsection 418.

The second optical position information producing section 420 is adaptedto calculate the position of the optical section 412 in the secondcoordination system 402 on the basis of a method which is the same asthe method of the second optical position information producing section120 of the first embodiment.

As shown in FIG. 32, the camera 410 mounted on the automotive vehicle408 in the second working space defines a camera coordination system 413having x-axis, y-axis, and z-axis on the basis of the second opticalposition information. The origin point of the camera coordination system413 is in register with the optical center of the optical section 412.The x-axis of the camera coordination system 413 extends in a horizontaldirection of the camera 410. The y-axis of the camera coordinationsystem 413 extends in a vertical direction of the camera 410. The z-axisof the camera coordination system 413 is in axial alignment with theoptical axis of the optical section 412.

The image coordination system 414 is defined on a plane spaced apartfrom the origin point of the camera coordination system 413 with a focallength “f” in the direction of the z-axis of the camera coordinationsystem 413. The image coordination system 414 has p-axis and q-axis. Thecamera 410 is adapted to obtain, as image information, an optical imagefocused on the image coordination system 414 through the optical section412.

The camera calibrating apparatus 400 further comprises an estimatedlocation information producing section 440 for producing estimatedlocation information of the bumper 409 of the automotive vehicle 408 tothe image coordination system 414 of the camera 410, and an estimatedlocation information storing section 450 for storing the estimatedlocation information produced by the estimated location informationproducing section 440.

The estimated location information producing section 440 is adapted toproduce estimated location information of the calibrating marker 406 tothe image coordination system 414 of the camera 410 from the positioninformation of the calibrating marker 406 stored in the vehicle positioninformation storing section 426 on the basis of the second opticalposition information produced by the second optical position informationproducing section 420. The calculation of the estimated location of thecalibrating marker 406 to the image coordination system 414 of thecamera 410 is performed on the basis of the above mentioned document 1.

As shown in FIG. 33, the bumper 409 of the automotive vehicle 408located on the second coordination system 402 is focused on the imagelocation Pn′ of the image coordination system 414 through the opticalsection 412. Here, the image location Pn′ is in register with theestimated image location Pn calculated by the estimated image locationcalculating section 440 under the condition that the housing unit 411 islocated on an accurate position of the automotive vehicle 408, morespecifically, the housing unit 411 is located at the positionrepresented by the second housing position information with no deviationto the accurate position. However, the housing unit 411, in general, islocated at an inaccurate position with deviation to the accurateposition. This leads to the fact that the second optical positioninformation is produced with deviation to the accurate position. As aresult, the estimated location Pn is spaced apart from the imagelocation Pn′ of the image coordination system 414.

The camera calibrating apparatus 400 further comprises a calibratingsection 460 for calibrating the second optical position informationstored in the second optical position information storing section 430 toorder to calibrate the above mentioned deviation of the second opticalposition information.

The calibrating section 460 is adapted to calibrate the second opticalposition information stored in the second optical position informationstoring section 430 on the basis of the image information of the bumper409 obtained by the camera 410 and the estimated location informationstored in the estimated location information storing section 450.

The calibrating section 460 includes an image location informationextracting section 470 for extracting image location informationindicative of an image location of the bumper 409 to the imagecoordination system 414 of the camera 410 on the basis of the imageinformation of the calibrating marker 406 obtained by the camera 110, acalibration value calculating section 480 for calculating a calibrationvalue of the second optical position information stored in the secondoptical position information storing section 430 on the basis of theimage location information extracted by the image location informationextracting section 470 and the estimated location information stored inthe estimated location information storing section 450, and an opticalposition information calibrating section 490 for calibrating the secondoptical position information stored in the second optical positioninformation storing section 430 on the basis of the calibration valuecalculated by the calibration value calculating section 480.

The calibration value calculating section 480 includes a superimposingsection 481 for superimposing one of a profile line of the bumper 409 ofthe image location information and a profile line of the bumper 409 ofthe estimated location information on the other of the profile line ofthe bumper 409 of the image location information and the profile line ofthe bumper 409 of the estimated location information, an extractingsection 482 for extracting two or more points, for example points of endportions of the bumper, from the profile lines of the automotive vehiclesuperimposed by the superimposing section 481, and a calculating section483 for calculating the calibration value of the second optical positioninformation stored by the second optical position information bycomparing the points of the image location information with the pointsof the estimated location information.

The superimposing section 481 is adapted to superimpose the profile line“P” (see FIG. 34(a)) of the bumper 409 of the estimated locationinformation on the profile line“P′” (see FIG. 34(b)) of the bumper 409of the image location information by allowing the profile line “P” ofthe bumper 409 of the estimated location information to be linearly androtationally moved with respect to the profile line “P′” of the bumper409 of the image location information (see FIG. 34(c)).

As shown in FIG. 35(a), the extracting section 482 is adapted to extracttwo or more points (P₁, P₂) and (P₁′, P₂′) from the profile lines “P”and “P′” of the bumper 409 superimposed by the superimposing section481.

The calculating section 483 is adapted to calculate a calibration valueof the second optical position information on the basis of a methodwhich is the same as the method of the calibration value calculatingsection 180 in the first embodiment by comparing the points (P₁, P₂)(see FIG. 35(b)) of the estimated location information with the points(P₁′, P₂′) (see FIG. 35(b)) of the image location information.

As shown in FIG. 36, the camera calibrating apparatus 400 thusconstructed as previously mentioned can be realized by comprising acomputer 491 for adjusting the camera 410, ECU 492 constituted as animage controlling apparatus for controlling the camera 410, and otherelements.

The computer 491 is shown in FIG. 26(a) as including CPU, RAM, ROM, aninput/output interface, and other elements. The computer 491 iselectrically connected to the camera 410 at the first working space. Inthis embodiment, the computer 491 constitutes each of the first housingposition information storing section 415, the second housing positioninformation storing section 416, the first optical position informationproducing section 417, the first optical position information storingsection 418, the second optical position information producing section420, the calibrating marker position information storing section 425,the calibrating marker position information storing section 426, thesecond optical position storing section 430, the estimated locationinformation producing section 440,. and the estimated locationinformation storing section 450.

The ECU 492 is shown in FIG. 36(c) as including CPU, RAM, ROM, aninput/output interface, and other elements. The ECU 492 is mounted onthe automotive vehicle 408 to be electrically connected to the camera410 in the second working space. In this embodiment, the ECU 492constitutes each of the second optical position information storingsection 430, the estimated location information storing section 450, andthe calibrating section 460.

As shown in FIG. 36(b), the camera 410 is transferred to the secondworking space from the first working space with a recordable medium 493such as for example CD-ROM and a magnetic disk. The recordable medium493 has stored therein each of the second optical position informationand the estimated location information to ensure that the each of thesecond optical position information and the estimated locationinformation is effectively transferred to the ECU 492 from the computer491.

In this embodiment, the camera 410 and the recordable medium 493 aretransferred to the second working space from the first working space,however, the camera 410, the second optical position information storingsection 430, and the calibrating section 460 may be collectivelytransferred as a camera unit 494 to the second working space from thefirst working space as shown in FIG. 37.

The following description will be directed to the operation of thefourth embodiment of the camera calibrating apparatus according to thepresent invention.

The second optical position information is calibrated by the cameracalibrating apparatus 400 through the steps shown in the FIG. 38.

The camera 410 is firstly located at the predetermined position on thefirst coordination system 401 in the first working space (in the stepS401). The first housing position information, the second housingposition information, the revising marker position information, and thevehicle position information are then stored in the first housingposition information storing section 415, the second housing positioninformation storing section 416, the revising marker positioninformation storing section 425, and the vehicle position informationstoring section 426, respectively (in the step S402). Here, the firsthousing position information, the second housing position information,the revising marker position information, and the vehicle positioninformation are obtained on the basis of positions predetermined in thedesign process and measured by a measuring instrument.

The image information of the revising marker 405 is then obtained by thecamera 410 (in the step S403). The first optical position information isthen produced by the first optical position information producingsection 417 from the revising marker position information stored in therevising marker position information storing section 425 on the basis ofthe image information of the revising marker 405 obtained by the camera410 (in the step S404). The first optical position information producedby the first optical position information producing section 417 is thenstored in the first optical position information storing section 418 (inthe step S405).

The second optical position information is then produced by the secondoptical position information producing section 420 from the secondhousing position information stored in the second housing positioninformation storing section 416 on the basis of the first housingposition information stored in the first housing position informationstoring section 415 and the first optical position information stored inthe first optical position information storing section 418 (in the stepS406). The second optical position information produced by the secondoptical position information producing section 420 is then stored in thesecond optical position information storing section 430 (in the stepS407).

The estimated location information indicative of the position of thebumper 409 to the image coordination system 414 of the camera 410 isthen produced by the estimated location information producing section440 from the vehicle position information stored in the vehicle positioninformation storing section 426 on the basis of the second opticalposition information produced by the second optical position informationproducing section 420 (in the step S408). The estimated locationinformation produced by the estimated location information producingsection 440 is stored in the estimated location information storingsection 450 (in the step S409).

The camera 410 and the recordable medium 493 are then transferred to thesecond working space from the first working space. The camera 410 isthen mounted to the automotive vehicle 408 to be located at thepredetermined position on the second coordination system 402 (in thestep S410).

The image of the bumper 409 is then taken by the camera 310 against thecalibrating plate 406 (in the step S411). As shown in FIG. 33, the imagelocation information of the bumper 409 is extracted from the imageinformation of the bumper 409 obtained by the camera 410 on the imagecoordination system 414 of the camera 410 (in the step S412).

As shown in FIG. 34, the profile line P′ of the bumper 409 of the imagelocation information is superimposed on the profile line P of the bumper409 of the estimated image information by the superimposing section 481(in the step S413).

As shown in FIG. 35, two or more points (P₁, P₂) and (P₁′, P₂′) of theoverlapped portion are then extracted from the profile lines “P” and“P′” of the bumper 409 by the extracting section 482 (in the step S414).The calibration value of the second optical position information is thencalculated by the calculating section 483 by comparing the points (P₁′,P₂′) of the image location information with the points (P₁, P₂) of theestimated location information (in the step S415).

The second optical position information stored in the second opticalposition information storing section 430 is finally calibrated by theoptical position information calibrating section 490 on the basis of thecalibration value calculated by the calibration value calculatingsection 480 (in the step S416). In this embodiment, the above mentionedsteps 401 to 415 may be executed as a program by a computer.

From the above detailed description, it will be understood that thecamera calibrating apparatus 400 can calibrate the second opticalposition information by using the motion vectors 388 a to 388 d.

In this embodiment, the camera calibrating apparatus 400 can calibratethe second optical position information by using the bumper 409 formingpart of the automotive vehicle 408.

In this embodiment, the camera calibrating apparatus 400 can accuratelydivide the plane-projected image by using the dividing marker 307.

As will be seen from the above detailed description, the cameracalibrating apparatus according to the present invention can calibratethe optical parameter of the camera mounted to the automotive vehicle.

1. A camera calibrating apparatus to be operative in combination with animaging device which includes a housing unit and an optical sectionsupported by said housing unit to obtain image information thorough saidoptical section, and adapted to calibrate position information of saidoptical section, comprising: first housing position information storingmeans for storing first housing position information indicative of aposition of said housing unit in the first coordination system; secondhousing position information storing means for storing second housingposition information indicative of a position of said housing unit inthe second coordination system; first optical position informationproducing means for producing first optical position informationindicative of a position of said optical section in said firstcoordination system on the basis of said image information obtained bysaid imaging device in said first coordination system; first opticalposition information storing means for storing said first opticalposition information produced by said first optical position informationproducing means; second optical position information producing means forproducing second optical position information indicative of a positionof said optical section to said second coordination system from saidsecond housing position information stored by said second housingposition information storing means on the basis of said first housingposition information stored by said first housing position informationstoring means and said first optical position information stored by saidfirst optical position information storing means; second opticalposition information storing means for storing said second opticalposition information produced by said second optical positioninformation producing means; and calibrating means for calibrating saidsecond optical position information stored by said second opticalposition information storing means on the basis of said imageinformation obtained by said imaging device in said second coordinationsystem.
 2. A camera calibrating apparatus to be operative in combinationwith an imaging device which includes a housing unit and an opticalsection supported by said housing unit to obtain image informationthorough said optical section, and adapted to calibrate optical positioninformation indicative of a position of said optical section,comprising: optical position information storing means for storingoptical position information indicative of a position of said opticalsection to a predetermined coordinate system; and calibrating means forcalibrating said optical position information stored by said opticalposition information storing means on the basis of said imageinformation obtained by said imaging device on said predeterminedcoordinate system.
 3. A camera calibrating apparatus to be operative incombination with an imaging device which includes a housing unit and anoptical section supported by said housing unit to obtain imageinformation thorough said optical section, and adapted to calibrateoptical position information indicative of a position of said opticalsection, comprising: first housing position information storing meansfor storing first housing position information indicative of a positionof said housing unit in the first coordination system in which arevising marker is located; second housing position information storingmeans for storing second housing position information indicative of aposition of said housing unit in the second coordination system in whicha calibrating marker is located; first optical position informationproducing means for producing first optical position informationindicative of a position of said optical section in said firstcoordination system on the basis of said image information of saidrevising marker obtained by said imaging device; first optical positioninformation storing means for storing said first optical positioninformation produced by said first optical position informationproducing means; second optical position information producing means forproducing second optical position information indicative of a positionof said optical section to said second coordination system from saidsecond housing position information stored by said second housingposition information storing means on the basis of said first housingposition information stored by said first housing position informationstoring means and said first optical position information stored by saidfirst optical position information storing means; estimated locationinformation producing means for producing estimated location informationindicative of a position of said calibrating marker to an imagecoordination system of said imaging device on the basis of said secondoptical position information produced by said second optical positioninformation producing means; second optical position information storingmeans for storing said second optical position information produced bysaid second optical position information producing means; estimatedlocation information storing means for storing said estimated locationinformation produced by said estimated location information estimatingmeans; and calibrating means for calibrating said second opticalposition information stored by said second optical position informationstoring means on the basis of said image information of said calibratingmarker obtained by said imaging device and said estimated locationinformation stored by said estimated location information storing means.4. A camera calibrating apparatus as set forth in claim 3, in which saidcalibrating means includes: image location information extracting meansfor extracting an image location information indicative of an imagelocation of said calibrating marker to said image coordination system ofsaid imaging device on the basis of said image information of saidcalibrating marker obtained by said imaging device; calibration valuecalculating means for calculating a calibration value of said secondoptical position information stored by said second optical positioninformation storing means on the basis of said image locationinformation extracted by said image location information extractingmeans and said estimated location information stored by said estimatedlocation information storing means; and optical position informationcalibrating means for calibrating said second optical positioninformation stored by said second optical position information storingmeans on the basis of said calibration value calculated by saidcalibration value calculating means.
 5. A camera calibrating apparatusas set forth in claim 4, in which said calibrating means is adapted tocalibrate a deviation of a rotational component of said second opticalposition information.
 6. A camera calibrating apparatus as set forth inclaim 4, in which said image location information extracting meansincludes: image displaying means for displaying an image of saidcalibrating marker obtained by said imaging device, and image locationspecifying means for specifying said image location of said calibratingmarker on said image of said calibrating marker displayed by said imagedisplaying means and to ensure that said image location information isextracted.
 7. A camera calibrating apparatus as set forth in claim 4, inwhich said image location information extracting means includes:estimated area information storing means for storing estimated areainformation indicative of said calibrating marker to said imagecoordination system of said imaging device; and image location searchingmeans for searching said image location of said calibrating marker fromsaid image information of said calibrating marker obtained by saidimaging device on the basis of said estimated area information stored bysaid estimated area information storing means and said estimatedlocation information stored by said estimated location informationstoring means to ensure that said image location information isextracted.
 8. A camera calibrating apparatus to be operative incombination with an imaging device which includes a housing unit and anoptical section supported by said housing unit to obtain imageinformation thorough said optical section, and adapted to calibrateoptical position information inductive of a position of said opticalsection, comprising: first housing position information storing meansfor storing first housing position information indicative of a positionof said housing unit in the first coordination system in which arevising marker is located; second housing position information storingmeans for storing second housing position information indicative of aposition of said housing unit in the second coordination system; firstoptical position information producing means for producing first opticalposition information indicative of a position of said optical section insaid first coordination system on the basis of said image informationobtained by said imaging device and indicative of said revising marker;first optical position information storing means for storing said firstoptical position information produced by said first optical positioninformation producing means; second optical position informationproducing means for producing second optical position informationindicative of a position of said optical section to said secondcoordination system from said second housing position information storedby said second housing position information storing means on the basisof said first housing position information stored by said first housingposition information storing means and said first optical positioninformation stored by said first optical position information storingmeans; second optical position information storing means for storingsaid second optical position information produced by said second opticalposition information producing means; and calibrating means forcalibrating said second optical position information stored by saidsecond optical position information storing means on the basis of amotion vector of said image information obtained by said imaging devicein said second coordination system.
 9. A camera calibrating apparatus asset forth in claim 8, in which said calibrating means includes:plane-projected image producing means for producing a plane-projectedimage from said image information obtained by said imaging device insaid second coordination system; plane-projected image dividing meansfor dividing said plane-projected image produced by said plane-projectedimage producing means into a plurality of image segments; motion vectorextracting means for extracting said motion vector from said imagesegments divided by said plane-projected image dividing means;calibration value calculating means for calculating a calibration valueof said second optical position information stored by said secondoptical position information storing means on the basis of said motionvector extracted by said motion vector extracting means; and opticalposition information calibrating means for calibrating said secondoptical position information stored by said second optical positioninformation storing means on the basis of said calibration valuecalculated by said calibration value calculating means.
 10. A cameracalibrating apparatus as set forth in claim 9, in which a dividingmarker located in said second coordination system in a predeterminedrelationship with said position of said housing unit represented by saidsecond position information stored by said second position informationstoring means, and in which plane-projected image dividing means isadapted to divide said plane-projected image produced by saidplane-projected image producing means into a plurality of image segmentson the basis of said image information of said dividing marker obtainedby said imaging device.
 11. A camera calibrating apparatus to beoperative in combination with an imaging device which includes a housingunit and an optical section supported by said housing unit to obtainimage information thorough said optical section, and adapted tocalibrate position information of said optical section, comprising:first housing position information storing means for storing firsthousing position information indicative of a position of said housingunit in the first coordination system in which a revising marker islocated; second housing position information storing means for storingsecond housing position information indicative of a position of saidhousing unit in the second coordination system in which an automotivevehicle is located; first optical position information producing meansfor producing first optical position information indicative of aposition of said optical section in said first coordination system onthe basis of said image information of said revising marker obtained bysaid imaging device; first optical position information storing meansfor storing said first optical position information produced by saidfirst optical position information producing means; second opticalposition information producing means for producing second opticalposition information indicative of a position of said optical section tosaid second coordination system from said second housing positioninformation stored by said second housing position information storingmeans on the basis of said first housing position information stored bysaid first housing position information storing means and said firstoptical position information stored by said first optical positioninformation storing means; estimated location information producingmeans for producing estimated location information indicative of aposition of said automotive vehicle to an image coordination system ofsaid imaging device on the basis of said second optical positioninformation produced by said second optical position informationproducing means; second optical position information storing means forstoring said second optical position information produced by said secondoptical position information producing means; estimated locationinformation storing means for storing said estimated locationinformation produced by said estimated location information estimatingmeans; and calibrating means for calibrating said second opticalposition information stored by said second optical position informationstoring means on the basis of said image information of said automotivevehicle obtained by said imaging device and said estimated locationinformation stored by said estimated location information storing means.12. A camera calibrating apparatus as set forth in claim 11, in whichsaid calibrating means includes: image location information extractingmeans for extracting an image location information indicative of animage location of said automotive vehicle in said image coordinationsystem of said imaging device on the basis of said image information ofsaid automotive vehicle obtained by said imaging device; calibrationvalue calculating means for calculating a calibration value of saidsecond optical position information stored by said second opticalposition information storing means on the basis of said image locationinformation extracted by said image location information extractingmeans and said estimated location information stored by said estimatedlocation information storing means; and optical position informationcalibrating means for calibrating said second optical positioninformation stored by said second optical position information storingmeans on the basis of said calibration value calculated by saidcalibration value calculating means.
 13. A camera calibrating apparatusas set forth in claim 12, in which said calibration value calculatingmeans includes: superimposing means for superimposing a profile line ofsaid automotive vehicle represented by said image location informationon a profile line of said automotive vehicle represented by saidestimated location information; extracting means for extracting aplurality of points from said overlapped profile lines of saidautomotive vehicle superimposed by said superimposing means, andcalculating means for calculating a calibration value of said secondoptical position information by comparing said points of said imagelocation information with said points of said estimated locationinformation.
 14. A camera calibrating apparatus as set forth in any oneof claims 1 to 13, in which said imaging device is mounted on anautomotive vehicle.
 15. An imaging system comprises a camera calibratingapparatus as set forth in any one of claims 1 to
 14. 16. An imagingcontrol system comprises a camera calibrating apparatus as set forth inany one of claims 1 to
 14. 17. A camera calibrating method ofcalibrating optical position information inductive of a position of anoptical section supported by a housing unit of a camera for obtainingimage information through said optical section, comprising: a firsthousing position information storing step of storing first housingposition information indicative of a position of said housing unit inthe first coordination system; a second housing position informationstoring step of storing second housing position information indicativeof a position of said housing unit in the second coordination system; afirst optical position information producing step of producing firstoptical position information indicative of a position of said opticalsection in said first coordination system on the basis of said imageinformation obtained by said imaging device in said first coordinationsystem; a first optical position information storing step of storingsaid first optical position information produced in said first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of a position of said optical section to saidsecond coordination system from said second housing position informationstored in said second housing position information storing step on thebasis of said first housing position information stored in said firsthousing position information storing step and said first opticalposition information stored in said first optical position informationstoring step; a second optical position information storing step ofstoring said second optical position information produced in said secondoptical position information producing step; and a calibrating step ofcalibrating said second optical position information stored in saidsecond optical position information storing step on the basis of saidimage information obtained by said imaging device in said secondcoordination system.
 18. A camera calibrating method of calibratingoptical position information inductive of a position of an opticalsection supported by a housing unit of a camera for obtaining imageinformation through said optical section, comprising: an opticalposition information storing step of storing optical positioninformation indicative of a position of said optical section to apredetermined coordinate system; and a calibrating step of calibratingsaid optical position information stored in said optical positioninformation storing step on the basis of said image information obtainedby said imaging device on said predetermined coordinate system.
 19. Acamera calibrating method of calibrating optical position informationinductive of a position of an optical section supported by a housingunit of a camera for obtaining image information through said opticalsection, comprising: a first housing position information storing stepof storing first housing position information indicative of a positionof said housing unit in the first coordination system in which arevising marker is located; a second housing position informationstoring step of storing second housing position information indicativeof a position of said housing unit in the second coordination system inwhich a calibrating marker is located; a first optical positioninformation producing step of producing first optical positioninformation indicative of a position of said optical section in saidfirst coordination system on the basis of said image information of saidrevising marker obtained by said imaging device; a first opticalposition information storing step of storing said first optical positioninformation produced in said first optical position informationproducing step; a second optical position information producing step ofproducing second optical position information indicative of a positionof said optical section to said second coordination system from saidsecond housing position information stored in said second housingposition information storing step on the basis of said first housingposition information stored in said first housing position informationstoring step and said first optical position information stored in saidfirst optical position information storing step; an estimated locationinformation producing step of producing estimated location informationindicative of a position of said calibrating marker to an imagecoordination system of said imaging device on the basis of said secondoptical position information produced in said second optical positioninformation producing step; a second optical position informationstoring step of storing said second optical position informationproduced in said second optical position information producing step; anestimated location information storing step of storing said estimatedlocation information produced in said estimated location informationestimating step; and a calibrating step of calibrating said secondoptical position information stored in said second optical positioninformation storing step on the basis of said image information of saidcalibrating marker obtained by said imaging device and said estimatedlocation information stored in said estimated location informationstoring step.
 20. A camera calibrating method of calibrating opticalposition information inductive of a position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough said optical section, comprising: a first housing positioninformation storing step of storing first housing position informationindicative of a position of said housing unit in the first coordinationsystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of a position of said housing unit in the second coordinationsystem; a first optical position information producing step of producingfirst optical position information indicative of a position of saidoptical section in said first coordination system on the basis of saidimage information obtained by said imaging device and indicative of saidrevising marker; a first optical position information storing step ofstoring said first optical position information produced in said firstoptical position information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of a position of said optical section to saidsecond coordination system from said second housing position informationstored in said second housing position information storing step on thebasis of said first housing position information stored in said firsthousing position information storing step and said first opticalposition information stored in said first optical position informationstoring step; a second optical position information storing step ofstoring said second optical position information produced in said secondoptical position information producing step; and a calibrating step ofcalibrating said second optical position information stored in saidsecond optical position information storing step on the basis of amotion vector of said image information obtained by said imaging devicein said second coordination system.
 21. A camera calibrating method ofcalibrating optical position information inductive of a position of anoptical section supported by a housing unit of a camera for obtainingimage information through said optical section, comprising: a firsthousing position information storing step of storing first housingposition information indicative of a position of said housing unit inthe first coordination system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of a position of said housingunit in the second coordination system in which an automotive vehicle islocated; a first optical position information producing step ofproducing first optical position information indicative of a position ofsaid optical section in said first coordination system on the basis ofsaid image information of said revising marker obtained by said imagingdevice; a first optical position information storing step of storingsaid first optical position information produced in said first opticalposition information producing step; a second optical positioninformation producing step of producing second optical positioninformation indicative of a position of said optical section to saidsecond coordination system from said second housing position informationstored in said second housing position information storing step on thebasis of said first housing position information stored in said firsthousing position information storing step and said first opticalposition information stored in said first optical position informationstoring step; an estimated location information producing step ofproducing estimated location information indicative of a position ofsaid automotive vehicle to an image coordination system of said imagingdevice on the basis of said second optical position information producedin said second optical position information producing step; a secondoptical position information storing step of storing said second opticalposition information produced in said second optical positioninformation producing step; an estimated location information storingstep of storing said estimated location information produced in saidestimated location information estimating step; and a calibrating stepof calibrating said second optical position information stored in saidsecond optical position information storing step on the basis of saidimage information of said automotive vehicle obtained by said imagingdevice and said estimated location information stored in said estimatedlocation information storing step.
 22. A camera calibrating method asset forth in any one of claims 17 to 21, in which said imaging device ismounted on an automotive vehicle.
 23. A camera calibrating program ofcalibrating optical position information inductive of a position of anoptical section supported by a housing unit of a camera for obtainingimage information through said optical section, and allowing a computerto execute: a first housing position information storing step of storingfirst housing position information indicative of a position of saidhousing unit in the first coordination system; a second housing positioninformation storing step of storing second housing position informationindicative of a position of said housing unit in the second coordinationsystem; a first optical position information producing step of producingfirst optical position information indicative of a position of saidoptical section in said first coordination system on the basis of saidimage information obtained by said imaging device in said firstcoordination system; a first optical position information storing stepof storing said first optical position information produced in saidfirst optical position information producing step; a second opticalposition information producing step of producing second optical positioninformation indicative of a position of said optical section to saidsecond coordination system from said second housing position informationstored in said second housing position information storing step on thebasis of said first housing position information stored in said firsthousing position information storing step and said first opticalposition information stored in said first optical position informationstoring step; a second optical position information storing step ofstoring said second optical position information produced in said secondoptical position information producing step; and a calibrating step ofcalibrating said second optical position information stored in saidsecond optical position information storing step on the basis of saidimage information obtained by said imaging device in said secondcoordination system.
 24. A camera calibrating program of calibratingoptical position information inductive of a position of an opticalsection supported by a housing unit of a camera for obtaining imageinformation through said optical section, and allowing a computer toexecute: an optical position information storing step of storing opticalposition information indicative of a position of said optical section toa predetermined coordinate system; and a calibrating step of calibratingsaid optical position information stored in said optical positioninformation storing step on the basis of said image information obtainedby said imaging device on said predetermined coordinate system.
 25. Acamera calibrating program of calibrating optical position informationinductive of a position of an optical section supported by a housingunit of a camera for obtaining image information through said opticalsection, and allowing a computer to execute: a first housing positioninformation storing step of storing first housing position informationindicative of a position of said housing unit in the first coordinationsystem in which a revising marker is located; a second housing positioninformation storing step of storing second housing position informationindicative of a position of said housing unit in the second coordinationsystem in which a calibrating marker is located; a first opticalposition information producing step of producing first optical positioninformation indicative of a position of said optical section in saidfirst coordination system on the basis of said image information of saidrevising marker obtained by said imaging device; a first opticalposition information storing step of storing said first optical positioninformation produced in said first optical position informationproducing step; a second optical position information producing step ofproducing second optical position information indicative of a positionof said optical section to said second coordination system from saidsecond housing position information stored in said second housingposition information storing step on the basis of said first housingposition information stored in said first housing position informationstoring step and said first optical position information stored in saidfirst optical position information storing step; an estimated locationinformation producing step of producing estimated location informationindicative of a position of said calibrating marker to an imagecoordination system of said imaging device on the basis of said secondoptical position information produced in said second optical positioninformation producing step; a second optical position informationstoring step of storing said second optical position informationproduced in said second optical position information producing step; anestimated location information storing step of storing said estimatedlocation information produced in said estimated location informationestimating step; and a calibrating step of calibrating said secondoptical position information stored in said second optical positioninformation storing step on the basis of said image information of saidcalibrating marker obtained by said imaging device and said estimatedlocation information stored in said estimated location informationstoring step.
 26. A camera calibrating program of calibrating opticalposition information inductive of a position of an optical sectionsupported by a housing unit of a camera for obtaining image informationthrough said optical section, and allowing a computer to execute: afirst housing position information storing step of storing first housingposition information indicative of a position of said housing unit inthe first coordination system in which a revising marker is located; asecond housing position information storing step of storing secondhousing position information indicative of a position of said housingunit in the second coordination system; a first optical positioninformation producing step of producing first optical positioninformation indicative of a position of said optical section in saidfirst coordination system on the basis of said image informationobtained by said imaging device and indicative of said revising marker;a first optical position information storing step of storing said firstoptical position information produced in said first optical positioninformation producing step; a second optical position informationproducing step of producing second optical position informationindicative of a position of said optical section to said secondcoordination system from said second housing position information storedin said second housing position information storing step on the basis ofsaid first housing position information stored in said first housingposition information storing step and said first optical positioninformation stored in said first optical position information storingstep; a second optical position information storing step of storing saidsecond optical position information produced in said second opticalposition information producing step; and a calibrating step ofcalibrating said second optical position information stored in saidsecond optical position information storing step on the basis of amotion vector of said image information obtained by said imaging devicein said second coordination system.
 27. A camera calibrating program ofcalibrating optical position information inductive of a position of anoptical section supported by a housing unit of a camera for obtainingimage information through said optical section, and allowing a computerto execute: a first housing position information storing step of storingfirst housing position information indicative of a position of saidhousing unit in the first coordination system in which a revising markeris located; a second housing position information storing step ofstoring second housing position information indicative of a position ofsaid housing unit in the second coordination system in which anautomotive vehicle is located; a first optical position informationproducing step of producing first optical position informationindicative of a position of said optical section in said firstcoordination system on the basis of said image information of saidrevising marker obtained by said imaging device; a first opticalposition information storing step of storing said first optical positioninformation produced in said first optical position informationproducing step; a second optical position information producing step ofproducing second optical position information indicative of a positionof said optical section to said second coordination system from saidsecond housing position information stored in said second housingposition information storing step on the basis of said first housingposition information stored in said first housing position informationstoring step and said first optical position information stored in saidfirst optical position information storing step; an estimated locationinformation producing step of producing estimated location informationindicative of a position of said automotive vehicle to an imagecoordination system of said imaging device on the basis of said secondoptical position information produced in said second optical positioninformation producing step; a second optical position informationstoring step of storing said second optical position informationproduced in said second optical position information producing step; anestimated location information storing step of storing said estimatedlocation information produced in said estimated location informationestimating step; and a calibrating step of calibrating said secondoptical position information stored in said second optical positioninformation storing step on the basis of said image information of saidautomotive vehicle obtained by said imaging device and said estimatedlocation information stored in said estimated location informationstoring step.